{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第五章 通过降维压缩数据\n",
    "\n",
    "在第四章，你学习了使用不同的特征选择方法来降维，除了特征选择，另一种降维的方法是特征抽取(feature extraction)。本章你将会学到三种基本的方法，帮助我们摘要数据集的信息，即将原始的特征空间压缩到低纬度的特征子空间。数据压缩是机器学习中的重要课题，它能帮助我们存储和分析当今时代不断涌现的大数据。本章，我们主要讨论以下几个内容：\n",
    "* 主成分分析（principal component analysis, PCA), 用于无监督数据压缩\n",
    "* 线性判别分析(linear discriminant analysis, LDA), 用于监督降维作为一种监督降维\n",
    "* 通过核PCA进行非线性降维\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## PCA进行无监督降维\n",
    "\n",
    "类似于特征选择，我们可以使用特征抽取来减小数据集中的特征维度。不过，不同于特征选择算法会保留原始特征空间，特征抽取会将原始特征转换/映射到一个新的特征空间。换句话说，特征抽取可以理解为是一种数据压缩的手段，同时保留大部分相关信息(译者注：理解为摘要)。特征抽取是用于提高计算效率的典型手段，另一个好处是也能够减小维度诅咒(curse of dimensionality)，特别是对于没有正则化的模型。\n",
    "\n",
    "PCA(principal component analysis, 主成分分析)是一种被广泛使用的无监督的线性转换技术，主要用于降维。其他领域的应用还包括探索数据分析和股票交易的信号去噪，基因数据分析和基因表达。\n",
    "\n",
    "PCA根据特征之间的相关性帮助我们确定数据中存在的模式。简而言之，PCA的目标是找到高维数据中最大方差的方向，并且将高维数据映射到一个新的子空间，这个子空间的方向不大于原始特征空间。新子空间的正交轴(主成分)可以被解释为原始空间的最大方差方向。下图中$x_{1}, x_{2}$是原始特征轴，$PC1和PC2$是主成分：\n",
    "\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/23/576c92f32c7a7.png)\n",
    "\n",
    "\n",
    "\n",
    "如果使用PCA降维，我们需要构造一个d*k维的转换矩阵$W$，它能将样本向量$x$映射到新的k维度的特征子空间，k<<d:\n",
    "\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/23/576c937fddbb3.png)\n",
    "\n",
    "\n",
    "映射后的子空间，第一主成分包含最大的方差，第二主成分包含次大的方差，以此类推，并且各个主成分方向是正交的(不相关)。\n",
    "\n",
    "PCA方向极其容易受到数据中特征范围影响，所以**在运用PCA前一定要做特征标准化**，这样才能保证每维度特征的重要性等同。\n",
    "\n",
    "\n",
    "再详细讲解PCA的细节之前，我们先介绍 PCA的步骤：\n",
    "\n",
    "\n",
    "* 1 将d维度原始数据标准化。\n",
    "* 2 构建协方差矩阵。\n",
    "* 3 求解协方差矩阵的特征向量和特征值。\n",
    "* 4 选择值最大的k个特征值对应的特征向量，k就是新特征空间的维度，k<<d。\n",
    "* 5 利用k特征向量构建映射矩阵$W$。\n",
    "* 6 将原始d维度的数据集X，通过映射矩阵W转换到k维度的特征子空间。\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 聊一聊方差\n",
    "\n",
    "本节，我们会学习PCA中的前四个步骤：标准化数据、构建协方差矩阵、得到特征值和特征向量以及对特征值排序得到排名靠前的特征向量。\n",
    "\n",
    "\n",
    "数据集还是用第四章介绍过的Wine数据集，先将原始Wine分割为训练集和测试集，然后标准化：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_wine = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/wine/wine.data', header=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python2.7/dist-packages/sklearn/cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.\n",
      "  \"This module will be removed in 0.20.\", DeprecationWarning)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.cross_validation import train_test_split\n",
    "from sklearn.preprocessing import StandardScaler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "X, y = df_wine.iloc[:, 1:].values, df_wine.iloc[:, 0].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "sc = StandardScaler()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_std = sc.fit_transform(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_test_std = sc.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面的代码完成了PCA的第一步，对数据进行标准化。我们再来看第二步：构建协方差矩阵。协方差矩阵是对称矩阵，d*d维度，其中d是原始数据的特征维度，协方差矩阵的每个元素是两两特征之间的协方差。\n",
    "\n",
    "举个例子，特征$x_{j}$和$x_{k}$的协方差计算公式如下：\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](https://ooo.0o0.ooo/2016/06/23/576ca3247088d.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中，$u_{j}和u_{k}$分别是样本中第j维度特征和第k维度特征的平均值。由于我们已经将数据标准化了，所以这里$u_{j}=u_{k}=0$。\n",
    "如果$\\sigma_{jk}>0$意味着j和k这两维度特征值要么同时增加要么同时衰减，反之$\\sigma_{jk}<0$,意味着这两个特征的值变化方向相反。\n",
    "\n",
    "\n",
    "假设数据有三维度特征，协方差矩阵如下：\n",
    "\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/23/576ca4622a37a.png)\n",
    "\n",
    "协方差矩阵的特征向量代表了主成分(最大方差的方向)，对应的特征值决定了特征向量绝对值的大小。在Wine数据集对应的13*13的协方差矩阵，我们会得到13个特征值。\n",
    "\n",
    "\n",
    "如何得到特征值和特征向量呢？会议线性代数课上讲过的内容：\n",
    "\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/23/576ca53ab284d.png)\n",
    "\n",
    "其中，$\\lambda$是特征值，一个常数。我们当然不可能手算特征值，NumPy提供了linalg.eig函数用于得到特征值和特征向量：\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "cov_mat = np.cov(X_train_std.T)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "eigen_vals, eigen_vecs = np.linalg.eig(cov_mat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Eigenvalues \n",
      "[ 4.8923083   2.46635032  1.42809973  1.01233462  0.84906459  0.60181514\n",
      "  0.52251546  0.08414846  0.33051429  0.29595018  0.16831254  0.21432212\n",
      "  0.2399553 ]\n"
     ]
    }
   ],
   "source": [
    "print(\"\\nEigenvalues \\n{}\".format(eigen_vals))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们先使用np.cov方法得到数据的协方差矩阵，然后利用linalg.eig方法计算出特征向量(eigen_vecs)和特征值(eigen_vals)。\n",
    "\n",
    "由于我们的目的是数据压缩，即降维，所以我们只将那些包含最多信息(方差)的特征向量(主成分)拿出来。什么样的特征向量包含的信息最多呢？这就要看特征值了，因为特征值定义了特征向量的大小，我们先对特征值进行降序排序，前k个特征值对应的特征向量就是我们要找的主成分。\n",
    "\n",
    "\n",
    "我们再学一个概念：方差解释率(variance explained ration)。一个特征值的方差解释率就是次特征值在特征值总和的占比：\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/23/576ca832336b3.png)\n",
    "\n",
    "\n",
    "\n",
    "利用NumPy提供的cumsum函数，我们可以计算累计解释方差和："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "tot = sum(eigen_vals)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "var_exp = [(i / tot) for i in sorted(eigen_vals, reverse=True)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "cum_var_exp = np.cumsum(var_exp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEKCAYAAAD5MJl4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3X2c1XP+//HHqwsSFV2wqZjsRpcz05hSSrXRhWVLlC59t6xYZH35srK+RF+7P98vK1u5IbFlV0iuYklSuSw1k6RLtYRpXaSIUBq9fn98PjNOc+biMzVnzpzpeb/dzu2cz+V5nU/TeZ335/35vN7m7oiIiMSqkewARESk6lFyEBGROEoOIiISR8lBRETiKDmIiEgcJQcREYmj5CAiInGUHEREJI6Sg4iIxKmV7ADKq3Hjxp6WlpbsMEREUkpubu4X7t4k6voplxzS0tLIyclJdhgiIinFzD4sz/o6rSQiInGUHEREJI6Sg4iIxFFyEBGROEoOIiISJ2HJwcweNLPPzWx1CcvNzCab2SYzW2VmWYmKRUREyieRLYcZQP9Slp8JtAofFwP3JDAWEREph4QlB3d/FdheyioDgYc8sBQ40syaJioeERGJLpk3wTUDPo6ZzgvnfZKccEREDtystz7imZVbyrVN22PrM+HX7RIU0f5JiTukzexiglNPHHfccUmORkRS2f58eZfHWx8EJ0xOadkwYe9RGZKZHLYALWKmm4fz4rj7NGAaQHZ2tic+NBGprp5ZuYW1n3xN26b1E7L/U1o2ZGBmM0ackto/ZJOZHOYC48zsUeAUYIe765SSiCRc26b1eeySrskOo0pLWHIws0eAXkBjM8sDJgC1Adz9XuB54FfAJuA7YEyiYhGR1JLIUz+JbDVUJwlLDu4+vIzlDlyeqPcXkdSVyFM/bZvWZ2Bmswrfb3WTEh3SInLw0amf5FJyEJFyS/QVPzr1k3yqrSQi5VZw2idRdOon+dRyEJH9otM+1ZtaDiIiEkctB5FqSH0CcqDUchCphtQnIAdKLQeRakp9AnIg1HIQEZE4Sg4iIhJHp5VEkkAdxlLVqeUgkgTqMJaqTi0HkSRRh7FUZWo5iIhIHCUHERGJo9NKIiXQgDNyMFPLQaQEiew0VoexVHVqOYiUQp3GcrBSy0FEROIoOYiISBwlBxERiaPkICIicZQcREQkjq5WkpSl4nUiiaOWg6QsFa8TSRy1HCSl6T4EkcRQy0FEROIoOYiISBwlBxERiaPkICIicZQcREQkjpKDiIjESWhyMLP+ZrbBzDaZ2fhilh9nZovM7G0zW2Vmv0pkPCIiEk3C7nMws5rA3UAfIA9YbmZz3X1tzGr/Dcx293vMrC3wPJCWqJikcukOZpHUVWbLwcyam9lTZrbVzD43syfMrHmEfXcGNrn7++7+A/AoMLDIOg4U/O9uAPy7PMFL1aY7mEVSV5SWw9+AWcCQcHpUOK9PGds1Az6Omc4DTimyzs3AfDO7AjgcOCNCPJJCdAezSGqK0ufQxN3/5u754WMG0KSC3n84MMPdmwO/Av5uZnExmdnFZpZjZjlbt26toLcWEZGSREkO28xslJnVDB+jgG0RttsCtIiZbh7Oi/VbYDaAuy8B6gCNi+7I3ae5e7a7ZzdpUlF5SUREShIlOVwInA98CnwCDAbGRNhuOdDKzFqa2SHAMGBukXU+Ak4HMLM2BMlBTQMRkSQrs8/B3T8EBpR3x+6eb2bjgBeBmsCD7r7GzCYCOe4+F/gv4H4zu4qgc3q0u3t530tERCpWicnBzP7g7v9nZlMIvrj34e6/L2vn7v48weWpsfNuinm9FuhWrohFRCThSms5rAufcyojEBERqTpKTA7u/mz48jt3fzx2mZkNKWYTERGpJqJ0SF8fcZ6IiFQTpfU5nElw70EzM5scs6g+kJ/owEREJHlK63P4N0F/wwAgN2b+N8BViQxKRESSq7Q+h3eAd8xslrvvqcSYpJKoMJ6IlCRKn0Oamc0xs7Vm9n7BI+GRScKpMJ6IlCRq4b0JwCTglwR3R2uQoGpChfFEpDhRvuQPc/eXAXP3D939ZuCsxIYlIiLJFKXlsDuslLoxLIexBTgisWGJiEgyRWk5XAnUBX4PnEwwnsNvEhmUiIgkV6kth3Coz6Hufg2wk2jVWEVEJMWV2nJw9x+B7pUUi4iIVBFR+hzeNrO5wOPAtwUz3f3JhEUlIiJJFSU51CEY+a13zDwHlBxERKqpKIP9qJ9BROQgo5vZREQkjpKDiIjEUXIQEZE4ZSYHMzvGzB4wsxfC6bZm9tvEhyYiIskSpeUwA3gRODacfg/4z0QFJCIiyRflUtbG7j7bzK4HcPd8M/sxwXEJGm9BRJInSsvhWzNrRHBvA2bWBdiR0KgE0HgLIpI8UVoOVwNzgZ+b2RtAE2BwQqOSQhpvQUSSIcpNcCvMrCdwEmDABg0bKiJSvUW5Wuly4Ah3X+Puq4EjzOyyxIcmIiLJEqXPYay7f1Uw4e5fAmMTF5KIiCRblORQ08ysYCIc4+GQxIUkIiLJFqVDeh7wmJndF05fEs4TEZFqKkpyuI4gIVwaTr8ETE9YRCIiknRRrlbaC9wTPkRE5CBQZnIws27AzcDx4foGuLufkNjQREQkWaKcVnoAuArIBVQ2Q0TkIBDlaqUd7v6Cu3/u7tsKHlF2bmb9zWyDmW0ys/ElrHO+ma01szVmNqtc0YuISEJEaTksMrPbCcaM3l0w091XlLZReMnr3UAfIA9YbmZz3X1tzDqtgOuBbu7+pZkdvR+fQUREKliU5HBK+JwdM8+B3mVs1xnY5O7vA5jZo8BAYG3MOmOBu8Mb63D3z6MELSIiiRXlaqVf7ue+mwEfx0zn8VOiKXAiQFjQryZws7vrHgqpMHv27CEvL49du3YlOxSRSlGnTh2aN29O7dq1D2g/UVoOmNlZQDugTsE8d594QO/80/u3AnoBzYFXzaxDbLmO8P0vBi4GOO644yrgbeVgkZeXR7169UhLSyPmRn+Rasnd2bZtG3l5ebRs2fKA9hWl8N69wFDgCoLLWIcQXNZali1Ai5jp5uG8WHnAXHff4+4fEIwy16rojtx9mrtnu3t2kyZNIry1SGDXrl00atRIiUEOCmZGo0aNKqSlHOVqpVPd/T+AL939FqAr4emgMiwHWplZSzM7BBhGMC5ErKcJWg2YWeNwv+9HjF0kEiUGOZhU1N97lOTwffj8nZkdC+wBmpa1kbvnA+MIxp9eB8x29zVmNtHMBoSrvQhsM7O1wCLg2qiXyYpIvM2bN9O+ffsy15k166erxnNycvj973+f6NDK5YgjjihznVNPPbVC3ivKMdtfFRVjMkTpc3jOzI4EbgdWEFypFKm2krs/DzxfZN5NMa+dYKS5q6MGLCIHpiA5jBgxAoDs7Gyys7PL2KrqefPNN5MdQony8/OpVatWlY6xLGW2HNz9f9z9K3d/gqCvobW735j40ESqh4ceeoj09HQyMjK44IILABg9ejRz5swpXKfgl/LixYvp2bMnAwcO5IQTTmD8+PE8/PDDdO7cmQ4dOvCvf/2r1O1jbd68mdNOO42srCyysrIKv6jGjx/Pa6+9RmZmJpMmTWLx4sWcffbZ7N27l7S0NL766qfrQVq1asVnn33G1q1bOe+88+jUqROdOnXijTfeiHu/H3/8kWuvvZZOnTqRnp7OffcFhZyfeuopTj/9dNydTz75hBNPPJFPP/2UGTNmMHDgQHr16kWrVq245ZZb4va5c+dOTj/9dLKysujQoQPPPPNMscesV69eDB48mNatWzNy5EiC352Qm5tLz549Ofnkk+nXrx+ffPJJ4fyMjAwyMjK4++67i/13GzZsGP/85z8LpwuOeUnHdfHixZx22mkMGDCAtm3b7hNjSZ9j8+bNtGnThrFjx9KuXTv69u3L998HJ2s2bdrEGWecQUZGBllZWYX/9rfffnvhMZ4wYUKxsVeEElsOZtbb3Rea2bnFLMPdn0xYVCIJcMuza1j7768rdJ9tj63PhF+3K3H5mjVruPXWW3nzzTdp3Lgx27dvL3Of77zzDuvWraNhw4accMIJXHTRRSxbtoy//vWvTJkyhbvuuitSbEcffTQvvfQSderUYePGjQwfPpycnBxuu+027rjjDp577jkg+FIDqFGjBgMHDuSpp55izJgxvPXWWxx//PEcc8wxjBgxgquuuoru3bvz0Ucf0a9fP9atW7fP+z3wwAM0aNCA5cuXs3v3brp160bfvn0ZNGgQTzzxBHfffTfz5s3jlltu4Wc/+xkAy5YtY/Xq1dStW5dOnTpx1lln7dOKqVOnDk899RT169fniy++oEuXLgwYMCDuvPrbb7/NmjVrOPbYY+nWrRtvvPEGp5xyCldccQXPPPMMTZo04bHHHuOGG27gwQcfZMyYMUydOpUePXpw7bXXFnv8hg4dyuzZsznrrLP44YcfePnll7nnnntw92KPK8CKFStYvXp13JVCJX0OgI0bN/LII49w//33c/755/PEE08watQoRo4cyfjx4xk0aBC7du1i7969zJ8/n40bN7Js2TLcnQEDBvDqq6/So0ePSH8T5VHaaaWewELg18Usc4I7pg96s976iGdWFr0Iq2Ks/eRr2jatn5B9S+VYuHAhQ4YMoXHjxgA0bNiwzG06depE06ZBt97Pf/5z+vbtC0CHDh1YtGhR5Pfes2cP48aNY+XKldSsWZP33nuvzG2GDh3KxIkTGTNmDI8++ihDhw4FYMGCBaxd+9P9q19//TU7d+7cp8Uyf/58Vq1aVdii2bFjBxs3bqRly5ZMmTKF9u3b06VLF4YPH164TZ8+fWjUqBEA5557Lq+//vo+ycHd+eMf/8irr75KjRo12LJlC5999llhcinQuXNnmjdvDkBmZiabN2/myCOPZPXq1fTp0wcIWjZNmzblq6++4quvvir8Qr3gggt44YUX4o7FmWeeyZVXXsnu3buZN28ePXr04LDDDmPHjh0lHtfOnTsXewlpSZ8DoGXLlmRmZgJw8skns3nzZr755hu2bNnCoEGDgCC5FBzj+fPn07FjRyBokWzcuLFyk4O7TzCzGsAL7j67wt+5mnhm5ZaEfYm3bVqfgZnNKny/B6vSfuFXtlq1arF3714A9u7dyw8//FC47NBDDy18XaNGjcLpGjVqkJ+fX+b2BSZNmsQxxxzDO++8w969ewu/YErTtWtXNm3axNatW3n66af57//+78L3WLp0aan7cHemTJlCv3794pbl5eVRo0YNPvvsM/bu3UuNGsEZ7aItgKLTDz/8MFu3biU3N5fatWuTlpZW7GWascesZs2a5Ofn4+60a9eOJUuW7LNu7Gmz0tSpU4devXrx4osv8thjjzFs2DCg9ON6+OGHF7uv0j5H0dgLTisVx925/vrrueSSSyJ9hgNRaoe0u+81sz8ASg6laNu0Po9d0jXZYUgV1Lt3bwYNGsTVV19No0aN2L59Ow0bNiQtLY3c3FzOP/985s6dy549e8q13yjb79ixg+bNm1OjRg1mzpzJjz8GRZXr1avHN998U+x+zaww3jZt2hT+qu/bty9TpkwpPAWzcuXKwl+7Bfr168c999xD7969qV27Nu+99x7NmjXj0EMP5cILL+SRRx5h5syZ3HnnnVxzzTUAvPTSS2zfvp3DDjuMp59+mgcffDDuMxx99NHUrl2bRYsW8eGHH0Y+RieddBJbt25lyZIldO3alT179vDee+/Rrl07jjzySF5//XW6d+/Oww8/XOI+hg4dyvTp08nJyWHGjBmlHtfSlPdz1KtXj+bNm/P0009zzjnnsHv3bn788Uf69evHjTfeyMiRIzniiCPYsmULtWvX5uijK74sXZRLWReY2TVm1sLMGhY8KjwSkWqoXbt23HDDDfTs2ZOMjAyuvjq4MG/s2LG88sorZGRksGTJkhJ/cZYkyvaXXXYZM2fOJCMjg/Xr1xeuk56eTs2aNcnIyGDSpElx2w0dOpR//OMfhaeUACZPnkxOTg7p6em0bduWe++9N267iy66iLZt25KVlUX79u255JJLyM/P589//jOnnXYa3bt3584772T69OmF/RWdO3fmvPPOIz09nfPOOy/uqqmRI0eSk5NDhw4deOihh2jdunXkY3TIIYcwZ84crrvuOjIyMsjMzCzsPP7b3/7G5ZdfTmZmZmHndXH69u3LK6+8whlnnMEhhxxS6nEtzf58jr///e9MnjyZ9PR0Tj31VD799FP69u3LiBEj6Nq1Kx06dGDw4MElJvoDZaUdGAAz+6CY2Ukb7Cc7O9sLOn+qgqH3BU1WtRyqpnXr1tGmTZtkhyHFmDFjBjk5OUydOjXZoVQ7xf3dm1muu0e+ZjlK4b0DK9AhIiIpJ2rhvfZAW/YtvPdQooISkepv9OjRjB49OtlhSAmijCE9gaD+UVuCu53PBF4HlBxERKqpKB3Sg4HTgU/dfQyQATRIaFQiIpJUkQrvufteIN/M6gOfs28pbhERqWai9DnkhIX37gdygZ3AktI3ERGRVBal8N5lYeG9e4E+wG/C00siEkF5yzYXFMIDmDt3Lrfddlup6990000sWLCg1P3sj7S0NL744ov93r4sRYsHFqekz7Y/evXqRSIug6/IGKuSKB3Sc4FHgWfcfXPCIxJJoEkvlV1fqDyu6lP2uFcHUrZ5wIABhQXaSjJxYkWM2Fs1VfXP9uOPP1b5GPdXlD6HvwDdgbVmNsfMBptZ2UVaRASIVlp63rx5tG7dmqysLJ588qealjNmzGDcuHHs2LGD448/vrCe0rfffkuLFi3Ys2fPPr/AS9rPzTffzB133FE43b59ezZv3gzAOeecw8knn0y7du2YNm1amZ9n/vz5dO3alaysLIYMGcLOnTvZsWMHJ510Ehs2bABg+PDh3H///YWf/6qrrqJdu3acfvrpbN26NW6fEydOpFOnTrRv356LL7648LjEfra0tDQmTJhQWPZ6/fr1hcfiwgsvpHPnznTs2LGwHPb333/PsGHDaNOmDYMGDSq2ZtG8efMYMmRI4XRsa+vSSy8lOzubdu3a7VMaOy0tjeuuu46srCwef/zxfWIs6XP06tWL6667js6dO3PiiSfy2muvAUFyueaaa2jfvj3p6elMmTIFKLnUeGWKclrpFXe/DDgBuA84n6BTWkTK6e233+auu+5i7dq1vP/++7zxxhvs2rWLsWPH8uyzz5Kbm8unn34at12DBg3IzMzklVdeAeC5556jX79+1K5du3CdKPspzoMPPkhubi45OTlMnjyZbdtKHozxiy++4NZbb2XBggWsWLGC7Oxs7rzzTho0aMDUqVMZPXo0jz76KF9++SVjx44Fgi/v7Oxs1qxZQ8+ePYsdt2HcuHEsX76c1atX8/333xeWEy+qcePGrFixgksvvbQw2f3pT3+id+/eLFu2jEWLFnHttdfy7bffcs8991C3bl3WrVvHLbfcQm5ubtz+zjjjDN566y2+/fZbgH0K7P3pT38iJyeHVatW8corr7Bq1arC7Ro1asSKFSsK143yOfLz81m2bBl33XVX4TGYNm0amzdvZuXKlaxatYqRI0eyZ88errjiCubMmUNubi4XXnghN9xwQ4n/JokSpeWAmR0GnAf8DugEzExkUCLVVUFp6Ro1ahSWll6/fj0tW7akVatWmBmjRo0qdtuhQ4fy2GOPAexTTrtA1P0UNXnyZDIyMujSpQsff/wxGzduLHHdpUuXsnbtWrp160ZmZiYzZ84sLCLXp08fOnTowOWXX8706T8NFlmjRo3CWEeNGsXrr78et99FixZxyimn0KFDBxYuXMiaNWuKff9zzw2GlykobQ1BS+a2224jMzOTXr16sWvXLj766CNeffXVwmOQnp5Oenp63P5q1apF//79efbZZ8nPz+ef//wnAwcOBGD27NlkZWXRsWNH1qxZs0/J8qLHPsrnKC72BQsWcMkll1CrVnCGv2HDhmzYsKGw1HhmZia33noreXl5xb5fIkXpc5gNdAbmAVOBV8JLW0WknIorLR3VgAED+OMf/8j27dvJzc2ld+/ekbeNLfENFJaLXrx4MQsWLGDJkiXUrVu38Mu1JO5Onz59eOSRR+KW7d27l3Xr1lG3bl2+/PLLwvEViipalnvXrl1cdtll5OTk0KJFC26++eYSYyg4frHHzt154oknOOmkk0o5AiUbNmwYU6dOpWHDhmRnZ1OvXj0++OAD7rjjDpYvX85RRx3F6NGj94mpuGJ7ZX2O4mIvTkmlxitblJbDA8DP3f137r5IiUGkYrVu3ZrNmzcXDgNZ3BcvBOfuO3XqxJVXXsnZZ59NzZo1I+8nLS2NFStWAMFoZR98ENTT3LFjB0cddRR169Zl/fr1LF26tNRYu3TpwhtvvMGmTZuA4JRRwWA3kyZNok2bNsyaNYsxY8YUlhHfu3dv4Tn5WbNm0b179332WfAF2rhxY3bu3FnmFUxF9evXjylTphSe33/77bcB6NGjB7NmzQJg9erV+5wWitWzZ09WrFjB/fffX3ia6Ouvv+bwww+nQYMGfPbZZ8UOBlTU/nyOPn36cN999xUmi+3bt+9TahyCQZtKakklUpQ+hxfdveyC5SKyX+rUqcO0adM466yzyMrKKrU2f3HltKPs57zzzmP79u20a9eOqVOncuKJwVVW/fv3Jz8/nzZt2jB+/Hi6dOlSaqxNmjRhxowZDB8+nPT0dLp27cr69evZsGED06dP5y9/+QunnXYaPXr04NZbbwWCX9nLli2jffv2LFy4kJtuummffR555JGMHTuW9u3b069fPzp16hT52AHceOON7Nmzh/T0dNq1a8eNNwZD3F966aXs3LmTNm3acNNNN3HyyScXu33NmjU5++yzeeGFFwo7ozMyMujYsSOtW7dmxIgRdOvWrcw49udzXHTRRRx33HGFY4zPmjWr1FLjlanMkt1VjUp2S3moZHfyHXHEEezcuTPZYRxUKqJkd6QOaRERObiU2CFtZlmlbejuKyo+HBGpbtRqSE2lXa30l/C5DpANvAMYkA7kADqPIiJSTZV4Wsndf+nuvwQ+AbLcPdvdTwY6AlsqK0CRA5Vq/WoiB6Ki/t6j9Dmc5O7vxrzxakA9fJIS6tSpw7Zt25Qg5KDg7mzbto06dQ68wlGUkt2rzGw68I9weiRQ/AXDIlVM8+bNycvLK7aej0h1VKdOnRJvQCyPKMlhDHApcGU4/SpwzwG/s0glqF27Ni1btkx2GCIpp8zk4O67zOxe4Hl331AJMYmISJKV2edgZgOAlQS1lTCzzHCMBxERqaaidEhPICi89xWAu68E1E4XEanGoiSHPe6+o8i8SJd+mFl/M9tgZpvMbHwp651nZm5mkW/tFhGRxImSHNaY2Qigppm1MrMpQJlVoMysJnA3cCbQFhhuZm2LWa8eQWf3W+WKXEREEiZKcrgCaAfsBh4Bvgb+M8J2nYFN7v6+u/9AMA71wGLW+x/gf4GSi8iLiEililKy+zt3v8HdO4V3Sd/g7lG+yJsBH8dM54XzCoX1m1q4+z/LFbWIiCRUlJHgTgSuAdJi13f36MNQFb/fGsCdwOgI614MXAxw3HHHHcjbiohIBFFugnscuBeYDpRn0J8tQIuY6ebsW5OpHtAeWBwOG/gzYK6ZDXD3fQZscPdpwDQIxnMoRwwiIrIfoiSHfHffnzuilwOtzKwlQVIYBowoWBheAdW4YNrMFgPXFE0MIiJS+aJ0SD9rZpeZWVMza1jwKGsjd88HxgEvAuuA2e6+xswmhjfWiYhIFRWl5fCb8PnamHkOnFDWhu7+PPB8kXk3lbBurwixJNwtz65h7b+/jrz+2k++pm3T+gmMSESk8kWprVRt7oae9NJ7Za7z9kdfsfWb3WWu1/yowwBo27Q+AzOblbG2iEhqKW2Y0N7uvtDMzi1uubs/mbiwkqfniU0irXdVnxMTHImISPKU1nLoCSwEfl3MMgeqZXIQEZFSkoO7Twifx1ReOCIiUhVE6ZDGzM4iKKFROPacu09MVFAiIpJcUcZzuBcYSlBjyYAhwPEJjktERJIoyn0Op7r7fwBfuvstQFdAvbEiItVYlOTwffj8nZkdC+wBmiYuJBERSbYofQ7PmdmRwO3ACoIrlaYnNCoREUmqKDfB/U/48gkzew6oU8zIcCIiUo2UdhNcsTe/hcuq7U1wIiJSesuhuJvfCugmOBGRaqy0m+B085uIyEEqyn0OjcxsspmtMLNcM/urmTWqjOBERCQ5olzK+iiwFTgPGBy+fiyRQYmISHJFuZS1acwVSwC3mtnQRAUkIiLJF6XlMN/MhplZjfBxPsHobiIiUk1FSQ5jgVnA7vDxKHCJmX1jZtGHTBMRkZQR5Sa4epURiIiIVB1Rrlb6bZHpmmY2IXEhiYhIskU5rXS6mT1vZk3NrD2wFFBrQkSkGotyWmlEeHXSu8C3wAh3fyPhkYmISNJEOa3UCrgSeAL4ELjAzOomOjAREUmeKKeVngVudPdLgJ7ARmB5QqMSEZGkinITXGd3/xrA3R34i5k9m9iwREQkmUpsOZjZHwDc/WszG1Jk8ehEBiUiIslV2mmlYTGvry+yrH8CYhERkSqitORgJbwublpERKqR0pKDl/C6uGkREalGSuuQzghrJxlwWEwdJQPqJDwyERFJmtJGgqtZmYGIiEjVEeU+BxEROcgoOYiISJyEJgcz629mG8xsk5mNL2b51Wa21sxWmdnLZnZ8IuMREZFoEpYczKwmcDdwJtAWGG5mbYus9jaQ7e7pwBzg/xIVj4iIRJfIlkNnYJO7v+/uPxCMIDcwdgV3X+Tu34WTS4HmCYxHREQiSmRyaAZ8HDOdF84ryW+BF4pbYGYXm1mOmeVs3bq1AkMUEZHiVIkOaTMbBWQDtxe33N2nuXu2u2c3adKkcoMTETkIRanKur+2AC1ippuH8/ZhZmcANwA93X13AuMREZGIEpkclgOtzKwlQVIYBoyIXcHMOgL3Af3d/fMExlIpJr30XoXt66o+J1bYvkREyithp5XcPR8YB7wIrANmu/saM5toZgPC1W4HjgAeN7OVZjY3UfGIiEh0iWw54O7PA88XmXfRMmHAAAAKLElEQVRTzOszEvn+IiKyf6pEh7SIiFQtSg4iIhJHyUFEROIoOYiISBwlBxERiaPkICIicZQcREQkjpKDiIjEUXIQEZE4Sg4iIhJHyUFEROIoOYiISBwlBxERiZPQqqxSsTRehIhUFrUcREQkjpKDiIjEUXIQEZE4Sg4iIhJHyUFEROIoOYiISBxdyiqFdKmsiBRQy0FEROKo5SCVQq0SkdSiloOIiMRRchARkThKDiIiEkd9DlItqE9DpGKp5SAiInHUchCJQC0TOdio5SAiInGUHEREJI5OK4lUAYk+baXTYlJeCU0OZtYf+CtQE5ju7rcVWX4o8BBwMrANGOrumxMZk4hULCW26ilhycHMagJ3A32APGC5mc1197Uxq/0W+NLdf2Fmw4D/BYYmKiYRkaKUfIqXyJZDZ2CTu78PYGaPAgOB2OQwELg5fD0HmGpm5u6ewLhERCpNqiafRHZINwM+jpnOC+cVu4675wM7gEYJjElERCKwRP1IN7PBQH93vyicvgA4xd3HxayzOlwnL5z+V7jOF0X2dTFwcTh5EkH/xD7rpJjGpG78qRw7pHb8qRw7pHb8qRw7BPEf7u5Nom6QyNNKW4AWMdPNw3nFrZNnZrWABgRf/Ptw92nAtIJpM8tx9+wKj7iSpHL8qRw7pHb8qRw7pHb8qRw7FMafVp5tEnlaaTnQysxamtkhwDBgbpF15gK/CV8PBhaqv0FEJPkS1nJw93wzGwe8SHAp64PuvsbMJgI57j4XeAD4u5ltArYTJBAREUmyhN7n4O7PA88XmXdTzOtdwJD92PW0slep0lI5/lSOHVI7/lSOHVI7/lSOHfYj/oR1SIuISOpSbSUREYmTcsnBzPqb2QYz22Rm45MdT1Rm1sLMFpnZWjNbY2ZXJjum8jKzmmb2tpk9l+xYysvMjjSzOWa23szWmVnXZMdUHmZ2Vfh3s9rMHjGzOsmOqTRm9qCZfR5erl4wr6GZvWRmG8Pno5IZY0lKiP328G9nlZk9ZWZHJjPG0hQXf8yy/zIzN7PGZe0npZJDTEmOM4G2wHAza5vcqCLLB/7L3dsCXYDLUyj2AlcC65IdxH76KzDP3VsDGaTQ5zCzZsDvgWx3b09wgUdVv3hjBtC/yLzxwMvu3gp4OZyuimYQH/tLQHt3TwfeA66v7KDKYQbx8WNmLYC+wEdRdpJSyYGYkhzu/gNQUJKjynP3T9x9Rfj6G4Ivp6J3jFdZZtYcOAuYnuxYysvMGgA9CK6Ow91/cPevkhtVudUCDgvvB6oL/DvJ8ZTK3V8luAIx1kBgZvh6JnBOpQYVUXGxu/v8sIoDwFKC+7aqpBKOPcAk4A9ApI7mVEsOUUpyVHlmlgZ0BN5KbiTlchfBH9beZAeyH1oCW4G/hafFppvZ4ckOKip33wLcQfCL7xNgh7vPT25U++UYd/8kfP0pcEwygzkAFwIvJDuI8jCzgcAWd38n6japlhxSnpkdATwB/Ke7f53seKIws7OBz909N9mx7KdaQBZwj7t3BL6l6p7SiBOemx9IkOSOBQ43s1HJjerAhDe7ptylkmZ2A8Ep4oeTHUtUZlYX+CNwU1nrxkq15BClJEeVZWa1CRLDw+7+ZLLjKYduwAAz20xwKq+3mf0juSGVSx6Q5+4FLbU5BMkiVZwBfODuW919D/AkcGqSY9ofn5lZU4Dw+fMkx1MuZjYaOBsYmWKVHH5O8MPinfD/cHNghZn9rLSNUi05RCnJUSWZmRGc817n7ncmO57ycPfr3b15WJtlGEGZk5T55erunwIfm9lJ4azT2bd0fFX3EdDFzOqGf0enk0Id6jFiy+X8BngmibGUSzhw2R+AAe7+XbLjKQ93f9fdj3b3tPD/cB6QFf6/KFFKJYewQ6igJMc6YLa7r0luVJF1Ay4g+NW9Mnz8KtlBHUSuAB42s1VAJvDnJMcTWdjimQOsAN4l+H9bpe/YNbNHgCXASWaWZ2a/BW4D+pjZRoLW0G2l7SNZSoh9KlAPeCn8v3tvUoMsRQnxl38/qdU6EhGRypBSLQcREakcSg4iIhJHyUFEROIoOYiISBwlBxERiaPkIEljZj+GlwWuNrPHwzs5i1vv+f2pgmlmx5rZnAOIb3OU6pWpzsxGm9mxyY5DqhYlB0mm7909M6w0+gPwu9iFFqjh7r/an0J57v5vdx9cUcFWY6MJynKIFFJykKriNeAXZpYWjtfxELAaaFHwCz5cts7M7g/HNphvZocBmNkvzGyBmb1jZivM7Ofh+qvD5aPN7BkzWxyOJzCh4I3N7Gkzyw33eXFZgVowpsiK8L1eDuc1DPezysyWmll6OP9mM5tpZq+Z2Ydmdq6Z/Z+ZvWtm88KSKgWtlIL5y8zsF+H8NDNbGO73ZTM7Lpw/w8wmm9mbZva+mQ2Oie9aM1sebnNLzH7ijl24XTbBDYIrw3m3WTDuyCozu6MC/m0lFbm7Hnok5QHsDJ9rEZRSuBRII6j82iVmvc1A43BZPpAZzp8NjApfvwUMCl/XIShrnQasDueNJqho2gg4jCDxZIfLGobPBfMbxb5vkZibEFQGbllk2ynAhPB1b2Bl+Ppm4HWgNsE4Et8BZ4bLngLOiXmvG8LX/wE8F75+FvhN+PpC4Onw9QzgcYIfeG0JStlDUK9/GmDhsucIypWXduwWxxyLRsAGfrpB9shk/53okZyHWg6STIeZ2Uogh6B+0APh/A/dfWkJ23zg7ivD17lAmpnVA5q5+1MA7r7Li69/85K7b3P37wmK13UP5//ezN4hqNPfAmhVSsxdgFfd/YPwvQrq5ncH/h7OWwg0MrP64bIXPCiY9y7BQD3zwvnvEnxpF3gk5rlgpLquwKzw9d9jYoYgUex197X8VP66b/h4m6DcRuuYzxN37Ir5fDuAXcADZnYuQTKTg1CtZAcgB7Xv3T0zdkZQV45vS9lmd8zrHwl+7UdVtFaMm1kvgjo/Xd39OzNbTNDyqEi7Adx9r5ntcfeCOPay7/9BL+F1qfsNWczz/3P3+2JXtGAMkTKPnbvnm1lnguJ+gwlqmfWOEItUM2o5SMrzYGS9PDM7B8DMDi3hyqc+Yd/AYQSjkL0BNAC+DBNDa4KWQWmWAj3MrGX4Xg3D+a8BI8N5vYAvvPzjdQyNeV4Svn6Tn4YEHRm+T2leBC60YNwQzKyZmR1dxjbfEBSVKxhvpIG7Pw9cRXAqTA5CajlIdXEBcJ+ZTQT2AEOIH7VuGcF4Gs2Bf7h7jpm9C/zOzNYRnGsv6XQWAO6+Ney0ftLMahCMSdCHoG/hQQuqvn7HT6Wpy+OocPvdwPBw3hUEI9hdSzCa3Zgy4ptvZm2AJWErbCcwiqClUJIZwL1m9j3B+OzPmFkdglbI1fvxOaQaUFVWOShYMFBLtruPS3YsxbFgEJZsd/8i2bGIgE4riYhIMdRyEBGROGo5iIhIHCUHERGJo+QgIiJxlBxERCSOkoOIiMRRchARkTj/H8vZkZmWIQybAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff3a99ea9d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.bar(range(1, 14), var_exp, alpha=0.5, align='center',\n",
    "       label='individual explained variance')\n",
    "plt.step(range(1, 14), cum_var_exp, where='mid',\n",
    "        label='cumulative explained variance')\n",
    "plt.ylabel('Explained variance ratio')\n",
    "plt.xlabel('Principal components')\n",
    "plt.legend(loc='best')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从上面的结果图我们可以看到第一个主成分占了近40%的方差(信息)，前两个主成分占了60%的方差。\n",
    "\n",
    "很多同学看到这里，可能将方差解释率和第四章讲到的用随机森林评估特征重要性联系起来，二者还是有很大区别的，PCA是一种无监督方法，在整个计算过程中我们都没有用到类别信息！随机森林是监督模型，建模时用到了类别信息。\n",
    "\n",
    "方差的物理含义是对值沿着特征轴的传播进行度量。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 特征转换\n",
    "\n",
    "在得到特征向量后，接下来我们就可以对原始特征进行转换了。本节我们先对特征值进行降序排序，然后用特征向量构建映射矩阵，最后用映射矩阵将原始数据映射到低维度特征子空间。\n",
    "\n",
    "先对特征值排序："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "eigen_pairs = [(np.abs(eigen_vals[i]), eigen_vecs[:, i]) for i in range(len(eigen_vals))]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "eigen_pairs.sort(reverse=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，我们选择最大的两个特征值对应的特征向量，这里只用两个特征向量是为了下面画图方便，在实际运用PCA时，到底选择几个特征向量，要考虑到计算效率和分类器的表现两个方面(译者注：常用的选择方式是特征值子集要包含90%方差)：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "w = np.hstack((eigen_pairs[0][1][:, np.newaxis],\n",
    "              eigen_pairs[1][1][:, np.newaxis]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Matrix W: \n",
      " [[ 0.14669811  0.50417079]\n",
      " [-0.24224554  0.24216889]\n",
      " [-0.02993442  0.28698484]\n",
      " [-0.25519002 -0.06468718]\n",
      " [ 0.12079772  0.22995385]\n",
      " [ 0.38934455  0.09363991]\n",
      " [ 0.42326486  0.01088622]\n",
      " [-0.30634956  0.01870216]\n",
      " [ 0.30572219  0.03040352]\n",
      " [-0.09869191  0.54527081]\n",
      " [ 0.30032535 -0.27924322]\n",
      " [ 0.36821154 -0.174365  ]\n",
      " [ 0.29259713  0.36315461]]\n"
     ]
    }
   ],
   "source": [
    "print(\"Matrix W: \\n {}\".format(w))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在，我们创建了一个13\\*2的的映射矩阵W。然后对样本(1*13维度)进行映射，就能得到2维度的新样本："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$x'=xW$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 2.59891628,  0.00484089])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train_std[0].dot(w)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "直接对原始数据(124\\*13)映射："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$X'=XW$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_pca = X_train_std.dot(w)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "特征维度降到2维度后，我们就可以用散点图将数据可视化出来了："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "colors = ['r', 'b', 'g']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "markers = ['s', 'x', 'o']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAEKCAYAAAAGvn7fAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3X+MXWWZB/Dv03ag7fZXmBZlKdMhG1db2jLS2pbI6tZaF1ikEXZNcHBFSCZk1c60bIzayAzZTNyssZ0hmphmIcviBONGWMEgWkBx16A4lUKLBWWbFkrqOq32B6GVae+zf5w5nXPvnHPPOfe855z3nPP9JBO4t/ee+9477XPe+5znfV5RVRARUXlMy3sARERkFgM7EVHJMLATEZUMAzsRUckwsBMRlQwDOxFRyTCwExGVDAM7EVHJMLATEZXMjDxedOHChdrZ2ZnHSxMRFdbu3buPquqisMflEtg7OzsxOjqax0sTERWWiByK8jimYoiISoaBnYioZBIHdhGZKSLPicgLIvKSiNxjYmBERNQaEzn2PwH4kKq+KSJtAP5HRH6gqj83cGwiIoopcWBXp6H7mxM32yZ+2OSdiCgnRnLsIjJdRPYA+D2AXar6C5/H9IjIqIiMjo2NmXhZIiLyYSSwq+o5Ve0CsBjAGhFZ7vOYnaq6WlVXL1oUWoZJRJSPefMAkak/8+blPbLIjFbFqOpxAD8GcK3J4xIRZebUqXj3W8hEVcwiEVkw8f+zAGwE8HLS4xIRUWtMVMVcAuABEZkO50TxHVX9voHjEhFRC0xUxbwI4L0GxkJERAZw5WkTI3tH0DnUiWn3TEPnUCdG9o7kPSQiolAM7AFG9o6g57EeHDpxCArFoROH0PNYT8vBnScJooKYOzfe/RZiYA+w7alteGv8rbr73hp/C9ue2hb7WKZPEkSUopMnAdWpPydP5j2yyBjYA7x24rVY9zdj8iRBRBSGgT1Ax/yOWPc3Y/IkQUQUhoE9wOCGQcxum1133+y22RjcMBj7WCZPEkREYRjYA3Sv6MbOj+7EkvlLIBAsmb8EOz+6E90rumMfK+pJghdYicgEcZozZmv16tVata3xRvaOYNtT2/DaidfQMb8DgxsG604S7gVWby5+dtvslk8mRFQ+IrJbVVeHPo6B3Q6dQ504dGLqdobts9px9PNHU3vdsBMOEdkjamBnKsYSQRdSj50+llpKhmWYROXEwG6JZhdS0yqLZBkmUQZyaAPMwG6JZtU2aZVFsgyTKAM5tAFmYLdE94putM9q9/2ztMoiWYZJVE4M7B55lxt+/IqPQyB197VaOx+FyVp9IrIHA/uEvC8kjuwdwQMvPAD17AMuEHzqyk+lVqVislafiOzBcscJQeWGS+YvwcG+g6V/fSJKiUjwn8WMvyx3jCmNC4lxUjtBr3PoxCGuRCUqshzaADOwTzB9ITFuaqfZ67DGnKjAcmgDzMA+wfSFxLg14oMbBqdcOG3EGnMiioKBfYLpC4lxUzvdK7rrLpzGPS4RkSvxZtYichmA/wDwDgAKYKeqDic9bh66V3Qnrghxe68EBelmKZcl85f4XkCN+nwiIsDMjP0sgLtUdRmAdQA+IyLLDBy3cLx5dT9hqR2/dFCc5xMRAQYCu6oeUdVfTfz/KQD7AVya9LhF1PuD3il5dVeU1E5jOqh9VjvaZ7WzxpyIYjFaxy4inQB+CmC5qp5s+LMeAD0A0NHRserQoeYph6IZ2TuCWx++1ffPBIJafy3jERGlYN48/x4nc+cWarPnosq8jl1E5gD4LoC+xqAOAKq6U1VXq+rqRYsWmXpZazSrVmFenEojTkOrsK6GOXQ9rAojgV1E2uAE9RFVfdjEMYumWbUK8+JUSWEngaRdD3liCJQ4sIuIALgPwH5V3Z58SOlIu8FX0Ky8fVY78+JEacihHW5RmJixvx/AJwF8SET2TPxcb+C4xmTR4CtogdPwdYWs/CQqp4rM8ivRBCyrBlvcP5RKL05Dq7DHJm2O1crzDTbkykPUi6eJFygVQVY7BZlY4ERktblzg1MdbtBkhUzuKtFSgDsFERnibWgVxA38YV0Nc+h6WBWVCOzcKYgoQJo557Cuhkm7HvLEEKgSgT2vnYLy3mqPKFSRK0tyaIdbFJW4eJoHtxLH22JgdttstgUguyS5mJjkuXmtYC34ylnuoJSzuP3YiSolr28KFZnlM7CnJKtKHKLcMMdtLQb2lLASh0qvIrPfImJgTwkrcagQyjLrbrXhWElXnjKwpySvShyiWGycdbcSaFttOBb2/IJiVQwRZS+oOsUrTmxK0r6gldfLCaticmBr3bqt4yJLJV20FOX57jcFU+OjOpUK7GkGuCw6SJZpXGSxZmmNKAE/7VLGkqVN0lCZVEzaC4ay6iAZl63jIou1MgP2xhGTHSD9xB0fUzHllfaCIVvr1m0dF1EmwhqONT6uJCoT2NMOcLbWrds6LiIA5sst4zYcs6EKKAWVCexBgeyiWRcZybvbWrdu67iIANhZblkClQnsfgHugukX4OSfThq5sGhr3bqt4yKLJU1LpL3oqSyLqlJUmYunwNSt6958+00cO31syuN4YZHIR5zOiAXvomirqBdPKxXYG027ZxoUU9+/QFDrr+UwIqKSKPjeorbKtCpGRO4Xkd+LyD4Tx8sKLywSURmZyrH/O4BrDR0rM7ywSGSBNLfnqygjgV1VfwrgDyaOlSVeWCSyQJG357PUjKxeSER6APQAQEdHvqmOxouogxsGGcyJqDQyC+yquhPATsC5eJrV6zYG8evfdT0eeOGB86tQ3RJHAAzuREHiVrnMnRv8eEqdsaoYEekE8H1VXR722KyqYvz6wwjEtxKGJY5ETaRZ5RKljwvLJAGwVwwA//4wfkEdYO8UIqu5s39eaI3EVLnjQwCeBfBuETksIneYOG5ScYI1SxypMmwLjnHSM7zQGomRHLuq3mLiOKZ1zO/wbVnbmI5hiSNVim3B0Zti4aYZRpQ6FRNUp37n6jtZ4khEpZVZVUwe3GBtY2kjSy4ryFT/lGb7hfIiI6HivWLykvZuTmQpU5UlYemKsGOZ3rXIZAwJO/lVvAcNq2IslvZuTkSFFdafnS17IylkYE9zU+osxNnNqejvlSxU5ODIjTkiKVyOvTGNUcSVo0HVOo0ll2V4r2QhBsHSK9yMvQxpjOvfdX2k+8vwXqkkijzLr6DCBfa0N6XOwuO/fTzS/WV4r+RhKjg2e3xagZYpkEIpXGC3eXOMqPnwqAHb5vdaea2s3jQVHIOOU+VAa9tq2pwVLrDbtDmGN5Av/NeF+PR/fTrSxthRA7ZN75Ua2LZ6s+r4+6hTuMBuy+YY7oVNN5AfO30M47XxuscE5cOjBmxb3iuRL86SrcUFSi3qHOr0rWxpJBA8eNODU1aZAmZWxHIFa06iLJQxtdLUVjYtFrJpLCmKukCJgb1F0+6ZFtgC2Kt9VjtOnz2dyipTrmDNUZRAUvZgY9P7s2ksKeLK05RFuYDpplvSKllkOWSJMK1BBjGwt8gvT37B9AvQPqsdAkH7rHbMmjELx04f832+iZJFlkPmyHRdd54X/8pwUmGdfR0G9hb5Xdi8f9P9OPr5o3jwpgdx+uzpwKAOhM/4o5ROshwyR2Wq6y5DRUmZfh8GMLAn0L2iGwf7DqLWXzu/X2rnUCduffjWKSkSr7CSxcaKm6DSSZZDUq44S7ZWoQO7TQ2yvMG4mSgli1Fz5yyHtFzegS/tFAtnydYqbFWMbRUhUcofl8xfcn5m30xQxY1AUOuvtTpEslkaVR1Rj1mRipIyyLQqRkSuFZFXRORVEfmCiWOGsa0iJOyCZZwUic25c5u+JZVK1Nl9GS50UuoSB3YRmQ7gGwCuA7AMwC0isizpccPYVhHSLOjGTZHYmjuPmvunFkRNa6RxoTPNlBFPRLkwMWNfA+BVVT2gqm8D+DaATQaO25Rts9qgYPytm76Fg30Hmwb1xm+7n1huZ+7ctm9JZEiKuXJtOOGc/6tepIqbAjIR2C8F8Lrn9uGJ+1Jl26y21QuZAwPAli2TwV3Vuf3b79ZX3OQd1AH7viWR3QYGgC3YcT6YK5zbA+jPcVTVkFlVjIj0iMioiIyOjY0lPp6NFSGN5Y9hY1EFjh8Hhocng/uWLc7t48ftu25l27ckCpFjVc75v9voOx/ct2AHhtGH41gQoRkHJZG4KkZErgYwoKp/M3H7iwCgql8Jek4ZesWY4g3mrt5eYMeO8M3os2ZbJVIl2VLB0qTBmZ44CZGJv9vThjCMvvN/3Ish7MAWCGDfzKUAMmsCJiIzAPwGwAYAbwD4JYBPqOpLQc9hYK+nCkzzfHeq1ewL6i7T3SRV699r421qYEvHyIBf0gD6cbx34PzEpCaC6Z75eQ2C889kYI8tamBPvJm1qp4Vkc8C+CGA6QDubxbUqZ47Y/fassXOGTvgpJtMzc4HBpyv6+57dT+LBQucPyMfFi/+UQDHseD8t8/t24FV0/YAnqUXW7DDmbFzdWq6VDXzn1WrVimp1mqqvb1OCUJvr//tsqryey+FgM35asD536P709Wleu4cf78mABjVCDE28YydWifizE69OfUdO5w/W7DAzhm7Kd73Ojw8eY3B1usLFI3A+f15rxnt3u2kGqvyd9sGhW0pkJUsdiiqcp65SNcXyCPgl6QAtvRqYDFAlf5up4EbbRiQ1UrLxr/oVfmLH3R9gdfUCsAnR64AtrR9A8PDTjCv1Zz/est5q/J3O28M7E1wpWV6vGWeQUGALOazWlVUseBL/zgltdjby/RL1phjbyLvlZZl3qi6ytcXymxgoH5m7v5e+fvMFgN7Ex3zO3xb8Wax0rJxMZCbBgJQmuDOIFBOVU0t2oSpmCby7EdTlTQQg0DK2F2xkhjYm8izH03eaSAqiTLsZ0qxMRUTwuRKyzjyTAMRUbFxxm4p29oSE1FxMLBbysa2xFQQ3rw6VRJXnpJ1qrwS14ioHxYXCxQOV55SIQXtKMVuj4axu2KpMbCTNYq2o1QrGt9D5u/J4H6mZC9WxZA1yt7xkf3nKSucsVMipmeg3uDuKkNQr8K3EbIHAzu1LI18eFk7PnobYg0PO62K3QZoxk9cOW5iTXZgYKeWpDEDLXvHx8y+jfh0XmRevVqYY6eWpJEPL3vHx1b2tzVR+sny0ephHTslksYOSGUMRI3fRnbsmHq78T2auNjKC7blkkkdu4j8vYi8JCI1EQl9MbJfnIuhaeXDy9jxMejbSNAmFCZSXbxgW2FRdrwO+gGwFMC7AfwEwOqoz1u1alV623hTy/r763eQr9Wc2/39Ux/r/pl313kTu9A3Pq9su9nHeX/ez9T9ifvZmjgG2QPAqEaIsYlm7Kq6X1VfSXx2odzFnd3FnYFGYfOqU1NlnXG+jZi42FrW8lEKESX6h/2AM/ZUZD17bWV2Z2qMaX0DMCHONxmTOGOnRog4Y48StJ8EsM/nZ5PnMaGBHUAPgFEAox0dHVl9DoWVZzDxBoEsA0BeQajZySmvE46J17X5ZEmtMRbYIx2EM3ajbAgmec3usj6xRDmB5vW5mDi55zVBoHQwsBdc1sHEhtmd6fccliaK857z+iZjItVV9gvSVZJJYAfwMQCHAfwJwP8B+GGU5zGwR2Pj7DUtpk8sUd9LlJOJDd9kiFQznrHH/WFgD2djvjltpk4scU8SzU6gNnyTIXIxsBdYlYNJGlU2zU6MUR7HPDXZImpgZ0sBS3EpeHKqzdsduJ9plGX+qtm0OcjqdaiYorYUYBMwSw0M1P+jdheapPWPvGwBxQ3aXo0Nt+I0HUuzzYH7Wbsn8+3bnRMST+bUKgZ2i2XVM6W/HzhxYjK41WrA1q3FDSjNZuJAfXBP+wQadsL0BnN35e8zzwA33uj8Ttz30OxEW7aTMiXHwF5x/f3Ao48Ce/Y4t7dvB1atcm6HBRRbxW3/m9YJNCydpp42DoDz2T/zjPPZu7+PsDbITNmRryiJeNM/vHhqB++Fw66u+guIXV2qZ89OfXyR5FnhE/UCuN/F26glrlW+yF5VYFUMRREUWO6+m5UgSTWruGm2+ClOiavfa2zePPX4VA5RAzu3xqs4EScF0OjRR9nHO6mgzorAZNdKv4u8XV3AuXPRtgX0ew0v9/hMy1QLc+wVV6s5OXWvri4nx9vVZW7buypSBfr66u9zb997L7B5s3P73nudz3b+/MnrHVu3Tp5wm7VB9jsx3Huv89+hofqLyEW8XkItijKtN/3DVIwdGnPs587V3/7yl6Pne6me97PdvNn58UuXNC5+cn8Hbsorbo698XWi5tvZT6YYwAVKFEVQ7fS8ec6m9u5sHeCMPS5vxQoQvFiqcSYdZ2btVxXjfgvwe62ox2BljZ2iLlDijJ2mzM68M3dWWyTjXihNs+9P44XSxll7s9diZU2xgFUxlAT7o5iRZeBs9bXYvbI4ogZ2pmIoUJIUAU3KMtXR6mtpSF8dskPUVAwDe4UwUOcny88+7mu5wZ/XU+wXNbCzjr0iBgbq66Hdf8ymZ4yN84Qc5g1WyqrvT9zX8gb13l5nph6lfp7sxsBeAerpSZLmgqOsTh5kTlBfnd7e5vXzZDcuUKoA7+rEtBYceU8ewNTe5kz72CvrFtGUPubYKyTsAlnSPHDauVpeI6CqY46d6rhB18ubNjGRRgnqjWIi+NqU5uF1BLJdosAuIl8VkZdF5EUReUREFpgaGJkTdoGsVjOTgw87ecQZr5d3fH199eP74x+zDaytnGB4IqCsJZ2x7wKwXFVXAvgNgC8mHxKZFnaBbNq0ydvDw85tv30/mzFVXeEXOLdudRpkrV3rLJV3x7d5s/OYe+5pPq5mt+No5SK0Td80qEKirGKK8gPgYwBGojyWK0/zEdboqbEveNyVh0lXqzZbObl5s+rnPlc/PnfpfNAqySQNtqKMMWyVZharTtm8q1qQdUsBAI8BuDXKYxnY7WNqWXnSQBM0jnPn/DsXNm4q4Xec3l5n4xB3lyj3eK22SIhzAkxzuT7bPlSPscAO4EkA+3x+Nnkesw3AI5iosgk4Tg+AUQCjHR0dmX0QFM62RlCNgdPblCyo/W3Y+3J/GtsTx31/rQTqpN+EwsZhw++MspHZjB3AbQCeBTA76nM4Y7ePLbO/oMB5993+fc3Xro0XVJPMnFsJpmnO2Nm8q3oyCewArgXwawCL4jyPgd1OeedrmwXOK6+czLG7AX7NmvhBNenMOc4JMKscu+lvA2SvqIE9aVXM1wHMBbBLRPaIyDcTHo9ylGU/k6DXb6ze2b7d2aLvhReA556brIS5915g3Trntt/Sd22o0jl3zjmOVyulmAMD9ZVCboWRX5VL2sv13ffoxf4uBMDcxdM4P5yxUzPNNv5oTDk0m6G6s+vGLf/uvrs+Z9+4UYXfGJLMhP3eT9JjM8deTWA/dioT1db6has6j/PbAvDqq53HPPtsfe9ydyPvNPqnm+zNzi3tqodb41FppFGK6d1CrnHG6y2LNDkTTmOWnfd1EcoWuDUelUGaKYdmNfOsZCEbRQ3sTMVQLtwUSdBtL780ypYtTpuBZu0Eoo7DL8XTauonyWu6f8YOlhSE3R3JWnH7pwwMOEF869bJQLd9O3DiRLJcsvu6Xm5TtLSqTYJeU5V9ZcgcBnbKlGr8RlqqThD3Pmfr1mQ7QHlft7Fp2apV6WwV1+w1+/qcTpVp73JFFRElX2P6hzn2aouaZw662GkqNx202OiDH0xvFW6zBU7Mv1MYMMdONgvLYfuV8vX1OQuTgp7T6jj8ctpp5rqbHTvN3D4VX9QcuzV7no6Pj+Pw4cM4c+ZM3kMJNHPmTCxevBhtbW15D6XQgvLM3iDeuH9qY1BvfE6rglbbprkKN+jYYZ8LUWRRpvWmf/xSMQcOHNCxsTGtWfq9s1ar6djYmB44cCDvoRRa1PJFb5td78+aNf416EXHlaQUBSKmYqyZsZ85cwadnZ0QS6cmIoL29naMjY3lPZRCC+qfAkz2T3Evlu7ZM/X569ZNXkhcu9ZMzxUbRPlciKKyJse+f/9+LF26NPOxxFWUcdouLIddqznVKd7gvnAh8LvfAXfdNbk13tBQuYJemrl9Kj7Wsbfg9ttvx8UXX4zly5fnPZTSa5bDdssZG2fsR48CM2ZMlguWLagD+XfYpHJgYPe47bbb8MQTT+Q9jMoTcRYkNbbZ9eIFRaJgxQzs8+Y5/6obf+bNS3TYD3zgA7jooosMDZKiaswG1mqTOfZWeqk33s/FPVQ11lw8jeXUqXj3k7X86tXdNExvr9M6wL3d1QXceOPkKlRg6sydrWyJihrYqRT86tW9S+7dpl9utYi3CRgwtVok7Hi8EElVwcBOufGW9A0PTwZkb8kf4My0vUHZfZ7fhcYoxyMqu2Lm2Kk0vMHYFRS0m92OezyiMmNg97jllltw9dVX45VXXsHixYtx33335T2k0gtaRt/qBU/TxyMqokSBXUT+WUReFJE9IvIjEflzUwNrau7cePdH9NBDD+HIkSPn+9bccccdiY5HzblB2FSLXNPHIyqqpDn2r6rqlwFARDYDuBvAnYlHFebkydRfgtJnehk9l+UTORIFdlX1Rtg/A8A5EcUS9cJoXscjKqLEVTEiMgjgHwCcALA+8Yiockwvo+eyfKq60By7iDwpIvt8fjYBgKpuU9XLAIwA+GyT4/SIyKiIjLJDIhFRekJn7Kr64YjHGgHwOID+gOPsBLATcLo7Rh0gERHFk7Qq5l2em5sAvJxsOERElFTSOvZ/mUjLvAjgIwB6DYwpN6+//jrWr1+PZcuW4YorrsCwu3SRiKhAklbF3GxqIPFf2/yGBDNmzMDXvvY1XHXVVTh16hRWrVqFjRs3YtmyZckOTESUoUKuPB0YqF9w4i5MSdq975JLLsFVV10FAJg7dy6WLl2KN954I9lBiYgyVrjA7u3g5wZ3d7Xh8ePmVhcePHgQzz//PNauXWvmgEREGSlcd8csOvi9+eabuPnmmzE0NIR5CTfvICLKWuFm7EC6HfzGx8dx8803o7u7GzfddFPyAxIRZayQgT2tDn6qijvuuANLly7F1q1bkx2MiCgnhQvsaXbw+9nPfoYHH3wQTz/9NLq6utDV1YXHH3/c3OCJiDJQyBx7Wh38rrnmGih7uxJRwRUusAPs4EdE1EzhUjEudvAjIvJX2MBORET+GNiJiEqGgZ2IqGQY2ImISoaB3ePMmTNYs2YNrrzySlxxxRXo7/fdM4SIyGqFLHdMy4UXXoinn34ac+bMwfj4OK655hpcd911WLduXd5DIyKKrLAz9pG9I+gc6sS0e6ahc6gTI3tHEh9TRDBnzhwATs+Y8fFxCOsoiahgChnYR/aOoOexHhw6cQgKxaETh9DzWI+R4H7u3Dl0dXXh4osvxsaNG9m2l3w1LlDmgmWySSED+7antuGt8bfq7ntr/C1se2pb4mNPnz4de/bsweHDh/Hcc89h3759iY9J5ZLWRi9EphQysL924rVY97diwYIFWL9+PZ544gljx6Tiy2qjF6IkChnYO+Z3xLo/qrGxMRw/fhwAcPr0aezatQvvec97Eh2TysXtS+R2FJ02bbLTKPsVkS2MBHYRuUtEVEQWmjhemMENg5jdNrvuvtltszG4YTDRcY8cOYL169dj5cqVeN/73oeNGzfihhtuSHRMKp80N3ohMiFxuaOIXAbgIwDM5UFCdK/oBuDk2l878Ro65ndgcMPg+ftbtXLlSjz//PMmhkglFrTRC4M72cJEHfsOAJ8H8D0Dx4qse0V34kBOFFfjRi87dkzeBhjcyQ6JAruIbALwhqq+wHpvqoI0N3ohMiU0sIvIkwDe6fNH2wB8CU4aJpSI9ADoAYCOjmQXOYnyxI1eyHahgV1VP+x3v4isAHA5AHe2vhjAr0Rkjar+zuc4OwHsBIDVq1f7FoWpqtUrPbltHrm40QvZrOVUjKruBXCxe1tEDgJYrapHWznezJkzcezYMbS3t1sZ3FUVx44dw8yZM/MeChFRU9Y0AVu8eDEOHz6MsbGxvIcSaObMmVi8eHHewyAiaspYYFfVziTPb2trw+WXX25oNERE1VXIladERBSMgZ2IqGQY2ImISkbyKOETkTEAh1p46kIALVXdVAg/o3D8jMLxMwqXx2e0RFUXhT0ol8DeKhEZVdXVeY/DZvyMwvEzCsfPKJzNnxFTMUREJcPATkRUMkUL7DvzHkAB8DMKx88oHD+jcNZ+RoXKsRMRUbiizdiJiChEIQN71lvxFYmIfFVEXhaRF0XkERFZkPeYbCEi14rIKyLyqoh8Ie/x2EZELhORH4vIr0XkJRHpzXtMthKR6SLyvIh8P++x+ClcYM9jK76C2QVguaquBPAbAF/MeTxWEJHpAL4B4DoAywDcIiLL8h2Vdc4CuEtVlwFYB+Az/IwC9QLYn/cgghQusGNyKz5eHPChqj9S1bMTN38Op08+AWsAvKqqB1T1bQDfBrAp5zFZRVWPqOqvJv7/FJzAdWm+o7KPiCwG8LcA/i3vsQQpVGD3bsWX91gK4nYAP8h7EJa4FMDrntuHwaAVSEQ6AbwXwC/yHYmVhuBMLmt5DySINf3YXaa24iuzZp+Rqn5v4jHb4Hy1HslybFR8IjIHwHcB9KnqybzHYxMRuQHA71V1t4j8dd7jCWJdYDe1FV+ZBX1GLhG5DcANADYo61ldbwC4zHN78cR95CEibXCC+oiqPpz3eCz0fgA3isj1AGYCmCci31LVW3MeV53C1rEn3YqvrETkWgDbAXxQVe3djipjIjIDzsXkDXAC+i8BfEJVX8p1YBYRZ8b0AIA/qGpf3uOx3cSM/Z9U9Ya8x9KoUDl2iuTrAOYC2CUie0Tkm3kPyAYTF5Q/C+CHcC4KfodBfYr3A/gkgA9N/N3ZMzEzpYIp7IydiIj8ccZORFQyDOxERCXDwE5EVDIM7EREJcPATkRUMgzsVAkicm6ifG+fiPyniMyeuP+dIvJtEflfEdktIo+LyF82PJddD6lQGNipKk6rapeqLgfwNoA7JxbkPALgJ6r6F6q6Ck43zHc0PJddD6lQrGspQJSB/wawEsB6AOOqen4Rl1+DOVU9AuDIxP/rU93yAAAAxUlEQVSfEhG36+GvsxkuUTycsVOlTLQWuA7AXgDLAeyO+fxOsOshWY6BnapilojsATAKZ5OW++IegF0PqSiYiqGqOK2qXd47ROQlAH8X5cnsekhFwhk7VdnTAC4UkR73DhFZKSJ/5X3QxEXW+wDsV9XtGY+RKDYGdqqsiV71HwPw4Ylyx5cAfAVAY39/dj2kQmF3RyKikuGMnYioZBjYiYhKhoGdiKhkGNiJiEqGgZ2IqGQY2ImISoaBnYioZBjYiYhK5v8BinbVi/QtJY8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff3a99ddd10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for l, c, m in zip(np.unique(y_train), colors, markers):\n",
    "    plt.scatter(X_train_pca[y_train==l, 0],\n",
    "               X_train_pca[y_train==l, 1],\n",
    "               c=c, label=l, marker=m)\n",
    "plt.xlabel('PC 1')\n",
    "plt.xlabel('PC 2')\n",
    "plt.legend(loc='lower left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从上图可以看到，数据在x轴(第一主成分)上要比y轴(第二主成分)分布更广，这也符合方差解释率的结果。数据降维后，直觉上使用线性分类器就能够将数据分类。\n",
    "\n",
    "\n",
    "### scikit-learn中的PCA\n",
    "\n",
    "上一小节我们详细讨论了PCA的步骤，在实际应用时，一般不会使用自己实现，而是直接调用sklearn中的PCA类，PCA类是另一个transformer类：我们先用训练集训练模型参数，然后统一应用于训练集和测试集。\n",
    "\n",
    "下面我们就是用sklearn中的PCA类对Wine数据降维，然后调用逻辑斯蒂回归模型分类，最后将决策界可视化出来："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "from matplotlib.colors import ListedColormap"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_decision_regions(X, y, classifier, resolution=0.02):\n",
    "    # setup marker generator and color map\n",
    "    markers = ('s', 'x', 'o', '^', 'v')\n",
    "    colors = ('red', 'blue', 'lightgreen', 'gray', 'cyan')\n",
    "    cmap = ListedColormap(colors[: len(np.unique(y))])\n",
    "    \n",
    "    # plot the decision surface\n",
    "    x1_min, x1_max = X[:, 0].min() - 1, X[:, 0].max() + 1\n",
    "    x2_min, x2_max = X[:, 1].min() - 1, X[:, 1].max() + 1\n",
    "    xx1, xx2 = np.meshgrid(np.arange(x1_min, x1_max, resolution),\n",
    "                          np.arange(x2_min, x2_max, resolution))\n",
    "    Z = classifier.predict(np.array([xx1.ravel(), xx2.ravel()]).T)\n",
    "    Z = Z.reshape(xx1.shape)\n",
    "    \n",
    "    plt.contourf(xx1, xx2, Z, alpha=.4, cmap=cmap)\n",
    "    plt.xlim(xx1.min(), xx1.max())\n",
    "    plt.ylim(xx2.min(), xx2.max())\n",
    "    \n",
    "    # plot class samples\n",
    "    for idx, cl in enumerate(np.unique(y)):\n",
    "        plt.scatter(x=X[y==cl, 0], y=X[y==cl, 1],\n",
    "                   alpha=.8, c=cmap(idx),\n",
    "                   marker=markers[idx], label=cl)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.decomposition import PCA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "pca = PCA(n_components=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr = LogisticRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_pca = pca.fit_transform(X_train_std)\n",
    "X_test_pca = pca.transform(X_test_std)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
       "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
       "          verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr.fit(X_train_pca, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3Xu0nHV97/H3d/YV2DukkqhZIQEVQdxcqoTAKkiVBCunKG29VDjWu0Ft64V29RzAHntWe6CnrgPqqe05UVzV1kOOntZqLbYlCTVpEEKgprgFaSgSgqbJRhL2TrIvM/M9fzzz7P3s2XOfZ+Z5ZubzysoiM3vmmV8G+H1+98fcHRERkUzSBRARkXRQIIiICKBAEBGRAgWCiIgACgQRESlQIIiICKBAEBGRAgWCiIgACgQRESnoT7oA9RgZWeGnnXZm0sUQEeko+/c/NOHuK6u9rqMC4bTTzuSWW/YkXQwRkY5yww32VC2v05CRiIgACgQRESlQIIiICKBAEBGRAgWCiIgACgQRESlQIIiICKBAEBGRAgWCiIgACgQRESlQIIiICKBAEBGRAgWCiIgACgQRESlQIIiICKBAEBGRgsQDwcz6zOyfzexbSZdFRKSXJR4IwEeBR5MuhIhIr0s0EMzsdOAXgS8kWQ4REUm+h/Bp4HeAfMLlEGkp98qPRdIgsUAws2uAQ+7+UJXXbTKzPWa2Z2rqcJtKJxKfHTtg69aFEHAPHu/YkWy5RIol2UO4DHiTmf0I2AJcaWZ/Ufwid9/s7uvcfd3IyMp2l1GkKe4wMwO7dy+EwtatweOZGfUUJF36k/pgd78JuAnAzF4L/La7vyOp8oi0ghls3Bj8effu4DfA+vXB82bJlU2kWNJzCCJdLxoKIYWBpFEqAsHd/9Hdr0m6HCKtEA4TRUXnFETSIrEhI5FeEJ0zCIeJwsegnoKkiwJBpIXMYGho8ZxBOHw0NKQwkHRRIIi02BVXBD2FsPIPQ0FhIGmTijkEkW5XXPkrDCSNFAgiIgIoEEREpECBICIigAJBREQKFAgiIgIoEEREpECBICIigAJBREQKFAgiIgIoEEREpECBICIigAJBREQKEgsEMxs2s91mttfMxs3svyZVFhERSfb46xngSnefMrMB4J/M7Nvufn+CZRIR6VmJBYK7OzBVeDhQ+K2bCoqIJCTROQQz6zOz7wGHgHvc/YEkyyMi0ssSDQR3z7n7zwKnA+vN7Lzi15jZJjPbY2Z7pqYOt7+QknrFN6vXzetFGpOKVUbufgS4F3hDiZ9tdvd17r5uZGRl+wsnqbZjR3DT+jAEwpva79hR/7UULNLrklxltNLMlhf+fBJwFfBYUuWRzuMOMzOwe/dCKGzdGjyemamvQo8zWEQ6VZKrjFYBXzKzPoJg+qq7fyvB8kiHCW9WD0EI7N4d/Hn9+vpuYh8NFgjeGwbL+vXBz3UPZOkFSa4y+hfgVUl9vnSHMBTCyhzqC4PoNaC5YBHpdKmYQxBpVDi0ExUd+om+rtLjaCiEFAbSaxQI0rGicwbr18PNNwf/DOcU8vngdeH8QPi41PxArcEi0s2SnEMQaYoZDA0tHtoJW/n798O2bbBhQzA/8MAD8PjjcP75C/MF4fwALA6W6BwCqKcgvUOBIB3tiisWT/qaBSGwbdtChb5hQxAGjz4aBMXoKFxyyeKKvlywDA0pDKR3KBCk4xVX2JkMDA7C8uWLJ4n7+oLXhhV+9H2lgkU9A+k1mkOQruMOs7Pw3HMwORk8d/gw5HLBz9zhnntKTyxXeizS7RQI0rHKrRwKh41+5mfg2DE4cABOnIDTToNbbw2GhrZtg82bO3fSWLuqpRUUCNKRKu0sdg8q/CNH4OSTF+YBpqeD5zt9o5l2VUurKBCk41Q7sgKCEFi+PJhAXrkSli0LAmH7dtizB668EjZtijcY2tFqj/O4DpFimlSWjlNtZzEEleNzzy2sJrrnHvjmNxfmEK66qrkwKO5l7NgRfGY4ER1W1ENDwYR1HJ8RPtauamkV9RCkI1XaWRzuTwjDIHTKKcEKJLPmNp0VD9nk8/DII0HvI65We6VhIe2qllZRD0E6UrmdxWHFGC4jDZ9/8MFgovmqq5rbdFbqILxt24LeyOmnx9Nqr3bYXj4ffGa5v7tIoxQI0nGKj6wot7M4rBz37w/mE8Lnw41q+/fXX4GWG7K55JLgurfdtvDaRivoSsNC0U132lUtcdOQkXScckdWrF+/dGexO6xdG7TgwxVGYYt+7drGhnNKDdmEFXVUM8NS5YaFMpna/+4i9VIPQTpSrTuLK7XoK7Wmo9eO7m+ApUM27vCnfxosc42r1V5pSKyeXdXlJqZFSlEgSMeqdWdxvfdMiK4Y2rkzWK4KMDwMl18eVP4HDgRLV8PKf/v2YA5hw4bmzkIqPmzv4otrm/co9RmtWPkk3S2xQDCzNcCXgRcBDmx2988kVR7pHsWt4LBFH31+69ag8s5klr43nNANK+dt24L3XXll8L4DB0pX/oODC9eLttprbaVHK/ChoSAMIAilegNGd4GTRiTZQ8gCv+XuD5vZKPCQmd3j7j9IsEwSk6SGKopbxfl80KLftw9e9jL48IeDCv6BB+CHP4QLLljcWi4eYgqfcw8eh8EQDZNKw1W1ttJLVeD33BOsjlq/fuG5Wr9D7VeQRiQ2qezuP3H3hwt/ngQeBVYnVR6JT1JHK5TaxbttGzz9NAwMBP8MewbLlwch8eij1e+eNjoa/A4r0XByt/g9tZSn3P6E6OTw7t3BmUthGBSvmqqV9itIvVIxh2BmZxLcX/mBEj/bBGwCeMEL1ra1XFK/JIcqyrWKr7wy+Nx7711Ysjk5GcwJvOQlpf8O0QndycnguTAUal3zX28rPY77Q1f6e4D2K0hliQeCmY0Afwl8zN2fL/65u28GNgOcccY6ndSScnENVTQ65FSqUr3qqoWf/c3fwPHjweM3vnHpERbRVnw4hh/OIYRDN/WsHqqnko+zAq91r4ZIVKL7EMxsgCAMvuLuf5VkWaQxpQ50a3aoopkhp2bvjRzd43DVVUEvYsOGoJcxPBw8V8+a/1rLU+3+0PXuZ6hnr4ZIKMlVRgbcCTzq7rcnVQ5pXKUJ0/DU0VCtLd1mhpzKtYrDCeSnnw7OMxodDYaBwr0Exb2E6Dr/6BEY0TmEesKgllZ6uQocGq/AdRc4qVeSQ0aXAb8GPGJm3ys8d7O7351gmaRGlSru5csXnzRaz1BFM0NO5SrV4hVCV10VrODZvh2efLL8tUr9udTjessDpSv5VlTgjZZdelNigeDu/wToP88OVaniHhwMbmHZaEu3mcnVUpVqOPQzPb3QGwjnFYaHW1tJ1lvJqwKXJCU+qSydq1LF3UxLt9nJ1VKVarmgaEeF28mVvI6+6GDRkxZrpECQhtVTcdcbBq1YHdPJFXMSdPRFhykRAJtuOg2AG26o7RIKBGlIqyruVkyuSv109EWHKAqBMAAapUCQhrSy4u7l1TGtHqKp9fo6+iKligPghX8N73tfbJdXIEjD6q2466nsenF4p9VDNPVeP+6d09KACsNAgfjCABQI0qRSFXepin/nTo1HV+IerIJ68MHgcfHhds30FMJ9FNFTXKNHape7vo6+SEjMw0D16KhAyJ7yUw6t2wLAY7e/XRVJCoWt0PA0UPegYnvySZiYCF6j8eildu4M/nnxxcH38sADwea5l7506a5vqP07i/YKwj0Z27YF5zqNjJQfAtLRF23U4mGgenRUICw/6SSuHRvj3v3jrP/kFg5NBs/PHF7Ok3/+BgVEwsKJyO3bg3sWf+hDQeXz4INw6qlw2mkajy4l/N4efDAIBPcgDI4fh5e8JKjUo/s6wvs7hL2rcuFQamIYguuefHLw83Lfvyb3W6hqACQTBtBhgRB63dqxRY/v3T/O0I1bOFR4/Njtbwc0FNFuZgs3sH/sseA8ntHRhZ3L69fDs88uvF5hEIjuqN6+HY4dC54PK+0wLCCojB95ZGEneHE4lLouLATx5GRw3VpObu3lyf1Y7dgBu3YteqqV8wDN6MhAKBYNiLD38PwkiwJC4dAemUzQM7j55qBiO3YsqFTCk0KjNB691LFjC+ctXXzxQq/h4osXhpGOH4dXvCII3/A473JDb9GJ4cnJ4PrhKa+1DAH14uR+LIp7AZeNd0QLtSsCIapU7yE6vAQKiFYKx6hHRxdaus8XDjWP3vAlzePRSe3OffLJhTAIXXxxcLzGa14TfH/hz44cWahzKg29RSeGzYLrhzQEFKOqAdAZFU7XBUKxUgHxCg0vtUR0InL58sVj4fffD5demv7x6CR254afMTGxcPhe9J4Ml1++uFIPT2sNw6FaGEQnhsOVS+G/g7SFcceoOg/QmRVK1wdCsVqGl0AB0YhwIjJ62umGDcE9jQ8cWPratFVG9ezOjbMXUW0CNzosFH6f//7vwc9HRysfF1J83fD8prQFceoVzQOkaSI4Tub13nkjQWMXjfmW+7a07Pr37h8H4PkeGF5q5bDId75TeVVMmkVb1aHiIZlW9SLK/TuJLuUNw2H5cjjvvOB7jvYAyvUUdEBdnUpNBHfIPEApdsMND7n7umqv67keQiW9MrzU6mGRn//5xZVOJtO+3kCzlV+13bm19iIaKUe5Cdzoap+wxR/d5wGVW/yaGK5Rl8wDNEOBUEE0IMZnxlnWBcNL7Tq0LIlKKI6gq7Y7t5YzfloRuKXCIXw+bUNvHePOO+HQofmH3TIP0IxEA8HMvghcAxxy9/OSLEs1Y0NjjK1deNypy1vbeWhZO4cq4gi6WnfnVupFtCNw1eJvUAftB0hK0j2EPwP+GPhywuWoWycPL7Xj0LJ2r9aJI+hq3Z1brRfRSaeEdvX8QpfNA7RDooHg7jvM7MwkyxCXThpeavWhZUmdpR9H0FUbkqm1F9HOU0IbrdS78gY4xcNAPTgP0IykewhVmdkmYBPAqjWrEi5NbYqHl4oD4tn7xzh83/mJ/E/XjkPLGmklx9FSjSvoKg3J1NKLaOcpoY1W6l1zA5yiAIDiYSAFQD1SHwjuvhnYDMGy04SL05BoQDyTfYbHR8c57dLx+d5DOwOi1mGROD6n1lZynJPB7Tids1Ivop3laKZS77ShrXlV5wGkGakPhG6zun81q9eunn9cKiBaPbzUjpUqtbaS42qptivoop9X6nGc5ajWa2q2Um/30FbDErw/QK9RICSsOCDGZ8Y560+2sG9fMLzUqqO9W7lSpZ5Wcpwt1bQsyYyjHLX2mpqp1FN7A5wU3R+g1yS97PQu4LXACjM7AHzS3e9MskxJGxsKJqfHCnPU4zPjDK3cksjwUqPqbSXH2VKNTv6GS0Sjj9ulmcCt9wiNRir1VN0Ap+p+AIVBuyS9yui6JD+/E4wNjc2HwzPZZzhy1jj7iuYfAM7tPz+ZApZRTys57pZqp6+eqbXX1Eyl3u4htkVKnQt0kwIgDaoGgpktA1a6+xNFz1/g7v/SspLJEqv7V7O6f/XSgNgHhxhn5vBypp5YnZpwqKWVHHdLtVtWz9TSa2q2Um/bEFupAHghkV6AAiAtKgaCmb0N+DRwyMwGgHe7e+HeTfwZ8OrWFk8qKQ6I8Zlx9q08wiGCQ/rSFhClxN1S7djVM0Vq7TXVW6n/0q0Xc9LkoSXPnxh9IX9984PxfD+lVgItGgZSAKRVtR7CzcBF7v4TM1sP/LmZ3eTuXwc65H+t3hEdXoKlAdENw0u16JjVM2XU22uqZ77ipMlDTI+sLPl8U3rkeOhuVy0Q+tz9JwDuvtvMXgd8y8zWAB25J6CXLAmIyPASBAGRlnCIc9VTalfP1CjR8f1aaRioK1ULhEkze1k4f1DoKbwW+GtgrOI7JXWiATE+M86y0XEOTY7P/zxNAdGoVK2eaUJaltDOUwD0hGqB8CGKhobcfdLM3gC8rWWlkpYrebxGJCA6Yf6hlI5oXdeolXtFAJYffJRMLhtc23Ncd9MZQGE+4fL/Ebyo7DCQAqAbVQuEY8CLgH1Fz68H7m9JiaQtJrIT7M/uZ9qnGbZh1vavZWztivmfl5p/SPv+h1DqWtcplcllyff1F/4M05wEU1OcdHQf7NqlXkAPqhYInwZuKvH884WfvTH2EknLTWQneHzucTJk6KefWZ/l8bnHAVjRH4RC8fDSWSX2P6S599Dq1nWnOjH6wvkJZMtnyeRyAOQtw8rpp4MaYWS4cDyEAqDXVAuEF7n7I8VPuvsj3XJsdS/an91Phgx91gdAH33gwfNhIESV2j3djfMPXW/HjmAoqDAMtGnq96C/H1asoC/6uqmpJEonKVAtEJZX+NlJcRak15UawilVOcdh2qfpL/pXnyHDtE/X9P5S8w8w3pHDS11tx47gn+Umg28agZGRZMomqVQtEPaY2Qfc/fPRJ83s/cBDrStWb6llCCdOwzbMrM8GPYOCPHmGbbih69UyvNTKgOjqu37Vo1oAABoGkkqqBcLHgK+b2X9kIQDWAYPAL7eyYL2k3iGcZq3tXxsEjgc9g3zh19r+tdXfXEWp4aVWzj90+rlFTSsKgboCYHQUJidLPx8jBXbnqBgI7v7vwM8VNqSdV3j6b919e8tL1kOaHcKpVxgy7RiiKhUQxcNL0Nju6W45t6gu5XoBl728kIB19ABuvjnespWweccrmJwZ4MaNj8wH9u1bz2d0aI5NVzzW8s+X+lQ7y2gY+CBwFvAIcKe7Z9tRsF4S9xBOVLnW2Yr+FS2bo6ik5PBS0e7pWoeXuuXcooo6aRjo1lsX9TjcYXL6w9yVeytwPjdufITbt57PXbvP4rr1+7ozsDtctSGjLwFzwE7gauBcgmEkiVGrhnD+3xd+hmNTfbzzoxPzrbMvf2YFp4zkeMv7n4up9I2LY3ip088tKqlTdwVPTi6apDbgxpEvwwTctfsj3LX7LNzh+kv2LeoxdPS/qy5TLRBe6e7nA5jZncDuKq+XBrRiCMcdjk318e2vngrAOz86wZc/s4Jvf/VUrn7b0VT+j9jI8FKnn1sElO8FpD0AamAGNw7/CXfxEQ5PDZPPGx/foOGjtKoWCHPhH9w9azH/H1Y4AuMzQB/wBXf/w1g/oIPEPYRjFoQAwLe/eup8MFz9tqPzPYa0qza8NPHdMR7etoKnHlrVWecWVZ0HgE4OgSh3uH36w/gI5PPGT49muPY31/CNkXdwx8yHuWt2PdcN3o3v/HPsltbPaUhl1QLhQjN7vvBnA04qPDbA3X1Zox9sZn3A54CrgAPAg2b2TXf/QaPXlMXCUAjDAEhVGNSz96Lc8NKTP3kZp5xxkAve8UMee2CMjRuD3kPqzi0qNQzUigAoGsefNzralknkKHe4ffID3DV7Dddfso+Pb3iEa39zDdtzP88Zk4/woswE141+kxtHv4xpM1wqVFtl1Ffp501aD+xz938DMLMtwLWAAiEm4ZxB1Jc/syIVodDs3oswIP7gd4O/5w9mYZ+Nc5hxLlgHP31gjEezCd77IalhoKJx/EXPt5kZjGamuG7wL7lx4xrM4Bsj7+CMyUfIkAeDG0c/n/h/i7IgyXsqrwaejjw+AFxS/CIz2wRsAli1ZlV7SlZBO3cUNyMMg3DOIDqHAMn3FOLce2FW4t7TL1+6eglaGBDlAiCt9wpuRU+ixL6GTXweXzmK2c24wx0zH+ZFmYn5M5Rvn/xAEAqNfaLELMlAqIm7bwY2A4xdNJboTXnavaO41OfXGkZmcMpIbtGcQTincMpILvFWWSv3XlS79zTEtDmuXcNArdCKnkSZIAnGl4MJ5Ltm1xeGiT4fDCcdfxMAN9odCoUUSDIQngHWRB6fXngutdq9oziqVBg9Ovsog3OD5MiVDIi3vP+5JcdAv/OjEzybm+Dh6WR7Oa3ce1GsVEDsaWRz3I4d8K//CocWbje56bLxzgiAhJnB6NAc1w3eHcwZFIaLIBhWUhikQ5KB8CDwcjN7CUEQvB24PsHyVNXuHcVRxWGUz+fJkiXnOU62k8v2Vop7As/mku3lhFp5fEY1q/tXs3psNVC+9wBw7n3PLQ2AJcNAbT4fo92TxjF+3qYrHgtWExUmkI1IzyDm4zKkMYkFQmEZ628Af0+w7PSL7j5e5W2JamertlhxGM0VVgQ7jpnV3FtJspcT1c7jMyopO7y05wh86rmgB/C+aKWfcC+g2lBP3OcTxTy0VLy0VD2DdEl0DsHd7wbuTrIM9UiyVVscRnnyQFCOUC29lVp7Oe2YPE/q+IxKwoA4tGacQ3cC+9rUA4irJd7mpaXSXVI/qZwmSbZqi8PIMPLkGbRBsp5l1mfJkaOffiayE2XLVEsvJ+nJ8zR43doxvvH0ruovjEu7l4u26qTTFO2DkPopEOqURKs2bK3nPEeWLIYxzDBZsuQ9zwwzWOFXhkzFyruWXk5ahpW6RhoryVZ9bpL7INL4PXcYBULKRVvrQzbEnM/Nzx8MMMA00/NBMMAAA5kBcp4rW3nX0stJcvK8K6Vos1hX0/fcNAVCykVb61nPMsssUJhDMMh7nmGGGcgMzL+nWuVdrZeT5OS5VNGmm9ok9nmSKAVCykVb67M+i2E4Tp48fdZHxjPMMssAC4GQJ08ffTw8/XBDcx1JTp5LFe0e+tBQS09RINQoqSMroq31PHmssFAvXF00wAAzzJDz3HzlnfUsXvhVblK40t8nLUtCe4pa4pICCoQaJLnqJtpaD3sHwHyPoM/6ONlPZtAG5ytvHDDKTgrX8vdJ45LQrtYtLXEFW0dTINQgyVU30db6nM/NLy3tt35yniNPnrMHz15UjvtO3Ie7M+Mz5MnPV/zTTCf+9+lJvVRJJhlsvfQ9t4gCoQZJr7qJttZrGbrKeIYTnJh/nCfPLLOc5CcByf99ek69lWSzyyfD9x89GpwqFzKDU0+Nbxlm2pZ5dksvK0EKhBqkadVNLUM5Zoa7L9rFnCdPeMe7NP19pIRml0+G7z96FPoj/4vncsHzcS3D1DLPrqNAqEGnrbrJkWOYYeaYmx8yGmaYHDmg8/4+0kZpa/VLWykQatBpq26GbZhZZjnZTp5/Luc5Bm0Q6Ly/T6LuvDNyl7Me0CmtfgVXSygQatRJq25q6QF00t8nMcMaQktUpUq/U4KrwygQYpSW22uqByBtc/BgMDcR5R5U5s221FXpt50CISbl1vY/n3ueI36k7RWzegAdrNnlk9H3Z7MLz5vB1FR8yzBHR+HIkaV3YervV6XdoRIJBDN7K/B7wLnAenffk0Q54lRqbX82n2V/bj/DNtyzx0hLA5ptWbdrDP3mm+Gmm0q34gt3RatbdJjoyJFgpRRAXx+8+MWNXVNqllQP4fvArwD/O6HPj12ptf05cvNnDoE2gEkHSHpzV3SY6OjRIAhg6bCUtEQigeDujwLz6+K7Qbm1/dG9AKANYJJynbJCJ+ng6lKaQ4hJqZU9hi06hRS0AUx6VCPLRPv6FnoG7gvDUFpa2jItCwQz2wqUGvS7xd2/Ucd1NgGbAFatWRVT6eJXamXPCzMv5GD+4KKTSLUBTLpKrS31RlYMRecMpqbgttsaK6PUrGWB4O4bY7rOZmAzwNhFY17l5YkqtbJnWXbZfEhkPJh0fnzucfZn92spqHQ+tdS7ioaMWiwMCd24XqQGmhtIVFLLTn8Z+J/ASuBvzex77v4LSZSlXXTktEgN1ONIVFKrjL4OfD2Jz26n6M7lWQ9ucxkGAmjFUSf4g9vewyf2JV2KFNOZQl1FQ0YtsmSIqPDL8sZAJlh51MiKo7Qcj5GWcrTStRddxDd27Uq6GOlRqvI/ciTYmVy8aSz6ultvDfYUHDmy+DVmcPrp8ZUFFERNUiC0SPEQ0RBDnPATzDJLv/c3tOIoLfMQaSmHtFmplUJHj1bfNDY5CatXL31+aqrxylvnHLWEAqFFincu91s/wz7MDDNkyTbUqk7LPERayiFdRq3+xCkQWqTUzuWMZTjVTuXVw69u6JppufVlWsohXSaOVv/Ro8H5SsXCVUoHDiy+rSgsDF0pdBQIrdKKu5Kl5daXaSmHyBLu1UOlv6jay+U01FSQqf4SacSK/hWcPXA2gzZIliyDNsjZA2c3NaSytn8tefLkPIe7k/NcIjuf01IOSYG+voVjJaK/tW+gI6mH0EJx35MgLTe+SUs5pM1KbRobGak+xl/LZrNbb1183HWor690i7/cNbvowMwkKBA6THHITGQneHj6Yd2AR+IX1yRvLa+dnAwq876+xc+XW8FU7pql5g+kZgqEDqbln9JS7V7aGT3dNOSu4ac2UiB0MC3/lK5S6o5o9e5VqDY8dfTo4tuKQtAzUegACoSOpuWfkjpJ7yXQ0tGmKBA6mJZ/SupoB3FHUyC0QLvO+WnFXgdZ6ncH+vn9uWz1F0rj2nXsddI9mJRTIMSsnRO9Wv7ZeusuOZNdT/046WIkI+5KuhWVcb3XVA+moo4PBM86ftBhJumSBEZ8hFfxKoxgPbTjHB84zoHTDrBiJP6KWss/pWVqraSjlXJ0L0Ff3+KJ4kqVcaNhoQo+Vp0fCAedFctWsPwFy7EUbEo5lj8GsBAI7kz+dBKeBUr8dyvS8aKV8tGjC3sJqp2CWu4axc9L2yR1x7RPAW8EZoEngPe4+5HK7ypjhtSEAQRB4CwcnmVmjLxghFMmTonl+r1wHwLpYNG9BOGRFlB++ElSJakewj3ATe6eNbP/DtwE/KdGL5aWMAAYtEFmfGZJKBQvD23ERHaCx2YfI0uWPHlmfZbHZh/jFbxCoSDpEB0impqC225beKxdxKmXyOF27v4P7h4u27gfaPC2Senwwfd9kDNefAbrLlhHv/UzZEPzQ0aGMWRDZKz5r/qJuSeYZRYIVhUBzDLLE3NPNH1tkZ4wOrr0ID4dxjcvDXMI7wX+b9KFaMY73vUObvj1G/jAuz8ABDfD6bf4v9rjfnw+aEKGcdyPx/5ZIrGrtGqp0eGkeldCaWlpRS0LBDPbCpTYi84t7v6NwmtuAbLAVypcZxOwCWDVmlVNlWlo/Wuww4eXPO8rVzKze2fD1738ist56kdPNVO0mhTPT0SfF0lMrZVypcq40iqjSlTBx6plgeDuGyv93MzeDVwDbHAvvoXRoutsBjYDjF14+mzkAAALDklEQVQ0VvZ1tbDDh/GVS8faS4VEGg0zzAlOLAoFxzGM+07cV3WSOZyQPpY/Nv++UzKnaGJamhNHpayKPRUSmUMwszcAvwO8yV3jHbU6a/As+ulf0iMYYGDRJriJ7MSS94Yb5o7nj5MlS44cc8xxIn+i7HtEpLckNYfwx8AQcE9hhdD97v7BhMrSdo0uHV3Rv4JzOXf+vVnP0kcfg5lBoPJpp/tm9zHLLHny889lyJAlyxBDOiFVRJIJBHc/K4nPTYNmj7aI7ky+78R9NZ12OpGd4DhLJ6Tz5DFMJ6RWMXTGLOzT3Wal+6VhlVHHe9f172Lnd3by7MSzvHzty/nEJz/Bu973rpKvjfMeBrWedhp+JrBkUjo8FE8npJa2un81e4YPwp1/Cu97X9LFkU7RoYfo9VQg+MqVZVcZNeNL/+dLNb82znsY1Hra6bRPM8ggM8wsWanUT79OSBWJW4cexdFTgdDM0tK4xHkPg1pPOx22YWaZZZjgs/Pk51cZnZQ5KdZVRjpaQ6Rz9VQgpEHc9zCo5bTT8DPdnfBXhgxr+9by0qGXNvS5pegezyKdTTNlbbaifwVnD5zNoA2SJcugDXL2wNktrTBX9K9gGcuYYWZ+lVGGDAfzB2NdbhqdHzEz+qyPDBn2Z/fH9hki0jrqISSg3fcwmMhOMOETWOEXQI4cfd4X63JT3eNZpLMpEHrA/uz+JauLDGOOOczjO/ZC93gWKWjXLUFjpkDoAdM+PT9fERV3Za17PIsUpHhpaSWaQ4jBgacPcPWGq7novItYd/46PvfZzyVdpEWGbXi+1e4UTSzHWFknMT8iIvHpuR6CO0Tvp1P8uBF9/X3c+qlbedWrX8Xk5CSXX3w5V268knNfeW5zF65DpeWeYct9wAfIkZvfoby2L/4lobrHs0jn6qlA+OLmPianjI98PItZEAafvaOf0RHnvZvquP9rkVWrVrFqVXA09+joKOe84hx+/MyPaw6EZtfuV1vuWet+BRHpbT0TCO4wOWV8bUswdPKRj2f57B39fG1LH299ey6WngLAUz96ir3f28vFl1xc0+vjWLtfy3EYarmLSDU9EwhmQQgAfG1L33wwvPXtufkeQ7Ompqa4/q3X80e3/xHLli2r6T1xnG2k5Z6ttWzlMH9w23v4xL6kSyLSWj01qRwNhVBcYTA3N8f1b7meX73+V7n2V66t+X3hCqCoeivzYRtu+QqiXva6tWNJF0GkLXqmhwALcwZRn72jv+lQcHc+9P4Pcc655/CRj3+krvfGsXZfyz1FYtKhp5TGpWd6CGEYhHMGOx+Y4a1vz/G1LX189o5+yt/Es7rv7voud/3FXXzn3u9w6asv5dJXX8rf3f13Nb13bf9a8uTJeQ53J+e5uitzLfcUiUl4Smnx75SfUhqXRHoIZvb7wLVAHjgEvNvdf9zaz4TREV80ZxAOH42OeFM9hJ+7/Oc4ljvW0HvjWgGkSWMRaVZSQ0afcvffBTCzjwD/BWj5LTTfu2nxaqIwFOKYQ2iGKnMRSYNEhozc/fnIw1OAJgZs6lNc+ScdBiIiaZHYpLKZ/TfgncBR4HVJlUNERAIt6yGY2VYz+36J39cCuPst7r4G+ArwGxWus8nM9pjZnucOP9eq4oqIBKuJpqaW/k75KaVxaVkPwd031vjSrwB3A58sc53NwGaAsYvG2ja0JCI9qAeWllaSyByCmb088vBa4LEkyiEiIguSmkP4QzM7h2DZ6VO0YYVRK01PT/P6176emZkZctkcv/TmX+ITv/eJpIslIlKXRALB3d+cxOe2ytDQEHdvvZuRkRHm5ubYeMVGXv+G17P+0vVJF01i9PG9o9xxYW9sUJLe1FNHVwA8Pfc0e2f2MpmfZDQzyoVDF7JmYE1T1zQzRkZGgOBMo7m5OUzrWbvKukvOZNdTP4a5pEsi0jo9c3QFBGGw68QujvtxhmyI436cXSd28fTc001fO5fLcemrL+XMF5/JlRuvrPn4axGRtOipQNg7s5eMZRiwAcyMARsgYxn2zuxt+tp9fX3c//D9PL7/cR568CHGvz8eQ4lFRNqnpwJhMj+55L4B/fQzlZ+K7TOWL1/OFa+9gnv+/p7Yriki0g49FQijmVGyLL4fQpYsI5mRpq57+PBhjhw5AsCJEyfYvnU755xzTlPXFBFpt56aVL5w6EJ2ndjFHHP000+WLHnPc+HwhU1d9+BPDrLpPZvI5XLk83ne/NY3c/U1V8dUahGR9uipQFgzsIbLuIy9M3uZyk8xkhnhwuHmVxmdf8H5fPeh78ZUShGRZPRUIEAQCs0GgIhIN+qpOQQRESlPgSAiIkCXBII3c0PkNkh7+UREoBsCYQiO/PRIaitdd+fIT4/AUNIlERGprOMnle3FxsTBCSYOTyRdlPKGgnJKZxs6Yxa++E9wxRVJF0WkJTo/EPoNO12VrbTW6v7V7Olr/swrkTTr/CEjERGJhQJBREQABYKIiBRYWlfnlGJmhwluudlOK4AUz1i3hb6DgL4HfQfQmd/BGe6+stqLOioQkmBme9x9XdLlSJK+g4C+B30H0N3fgYaMREQEUCCIiEiBAqG6zUkXIAX0HQT0Peg7gC7+DjSHICIigHoIIiJSoECog5n9lpm5ma1IuiztZmafMrPHzOxfzOzrZrY86TK1i5m9wcx+aGb7zOw/J12edjOzNWZ2r5n9wMzGzeyjSZcpKWbWZ2b/bGbfSrosraBAqJGZrQFeD+xPuiwJuQc4z90vAB4Hbkq4PG1hZn3A54CrgVcC15nZK5MtVdtlgd9y91cClwK/3oPfQeijwKNJF6JVFAi1uwP4HaAnJ13c/R/cPVt4eD9wepLlaaP1wD53/zd3nwW2ANcmXKa2cvefuPvDhT9PElSIq5MtVfuZ2enALwJfSLosraJAqIGZXQs84+57ky5LSrwX+HbShWiT1UD0mNMD9GBlGDKzM4FXAQ8kW5JEfJqgUZhPuiCt0vHHX8fFzLYCLy7xo1uAmwmGi7pape/A3b9ReM0tBEMIX2ln2SR5ZjYC/CXwMXd/PunytJOZXQMccveHzOy1SZenVRQIBe6+sdTzZnY+8BJgr5lBMFTysJmtd/eDbSxiy5X7DkJm9m7gGmCD98565WeANZHHpxee6ylmNkAQBl9x979KujwJuAx4k5n9B2AYWGZmf+Hu70i4XLHSPoQ6mdmPgHXu3mmHWzXFzN4A3A78vLsfTro87WJm/QST6BsIguBB4Hp3H0+0YG1kQUvoS8BP3f1jSZcnaYUewm+7+zVJlyVumkOQWv0xMArcY2bfM7P/lXSB2qEwkf4bwN8TTKZ+tZfCoOAy4NeAKwv/7r9XaClLl1EPQUREAPUQRESkQIEgIiKAAkFERAoUCCIiAigQRESkQIEgUgMzyxWWW37fzL5mZicXnn+xmW0xsyfM7CEzu9vMzi787O/M7Ei3nowp3UeBIFKbE+7+s+5+HjALfLCwYevrwD+6+8vc/SKCU2BfVHjPpwjW74t0BAWCSP12AmcBrwPm3H1+k56773X3nYU/bwMmkymiSP0UCCJ1KBxlcTXwCHAe8FCyJRKJjwJBpDYnmdn3gD0EN0m6M+HyiMROp52K1OaEu/9s9AkzGwfeklB5RGKnHoJI47YDQ2a2KXzCzC4ws9ckWCaRhikQRBpUuCfELwMbC8tOx4HbgIMAZrYT+BqwwcwOmNkvJFdakep02qmIiADqIYiISIECQUREAAWCiIgUKBBERARQIIiISIECQUREAAWCiIgUKBBERASA/w+mE8qc8GbvbgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff3a79b4310>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_decision_regions(X_train_pca, y_train, classifier=lr)\n",
    "plt.xlabel('PC1')\n",
    "plt.ylabel('PC2')\n",
    "plt.legend(loc='lower left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果你仔细观察我们自己实现的PCA得到的散点图和调用sklearn中PCA得到的散点图，两幅图看起来是镜像关系！这是由于NumPy和sklearn求解特征向量时计算的差异，如果你实在看不惯，只需要将其中一个得到的特征向量*(-1)即可。还要注意特征向量一般都要归一化。\n",
    "\n",
    "我们再看看决策界在测试集的分类效果："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3X9w3PV95/HnW1pZwpFcJ9iJPbId54DwQxhCcAwTp24AN4UrrZsmtEDphSaNINNMft5kDkgvN9M76DQdkssl16snzpQ2HJ5yDeWGkqQYSExM+GE7OM4CdZQe2Di4tpI4yGD9WO37/thdebVaaX/v57vf7+uR8aBdrb/7RhHf135+m7sjIiLSFboAERGJBgWCiIgACgQREclTIIiICKBAEBGRPAWCiIgACgQREclTIIiICKBAEBGRvFToAmrR37/MTz99begyREQ6ysGDe0bdfXml13VUIJx++lpuu2136DJERDrKTTfZi9W8Tl1GIiICKBBERCQveCCYWbeZ/cDMHghdi4hIkgUPBODjwHOhixARSbqggWBmq4DfBL4asg4REQnfQvgi8BkgO98LzGzYzHab2e4TJ461rzIRkYQJFghmdjVw1N33LPQ6d9/q7uvdfX1/f8VptCIiUqeQLYSNwG+b2QvAduByM/t6wHpERBItWCC4+y3uvsrd1wLXAo+4+w2h6hERSbrQYwgiIhIRkdi6wt2/A3wncBkiIommFoKIiAAKBBERyVMgiIgIoEAQEZE8BYKIiAAKBBERyVMgiIgIoEAQEZE8BYJITLkv/FiklAJBJIZ27oQdO06FgHvu8c6dYeuSaFMgiMSMO0xMwFNPnQqFHTtyjycm1FKQ+UViLyMRaR4z2Lw59/VTT+X+AGzYkHveLFxtEm1qIYjEUHEoFCgMpBIFgkgMFbqJihWPKYiUoy4jkZgpHjModBMVHoNaCjI/BYJIzJhBb+/sMYNC91Fvr8JA5hcsEMysD9gJ9Obr+D/u/rlQ9YjEyaZNuZZC4eZfCAWFgSwkZAthArjc3U+YWQ/wPTP7prs/EbAmkdgovfkrDKSSYIHg7g6cyD/syf/RkJeISCBBZxmZWbeZPQMcBR5y9yfLvGbYzHab2e4TJ461v0gRkYQIGgjuPu3ubwNWARvM7Pwyr9nq7uvdfX1///L2FykikhCRWIfg7seBR4ErQ9ciIpJUwQLBzJab2dL816cBvw48H6oeEZGkCznLaCVwl5l1kwumv3f3BwLWIyKSaCFnGf0QuCjU+4uIyGyRGEMQEZHwFAgiIgIoEEREJE+BICIigAJBRETyFAgiIgIoEEREJE+BICIigAJBEq70jOGFzhyu5bUinUiBIIm1c+fsg+cLZxHv3Fn5tdns/K8V6VQKBEkkd5iYyB08X7jRFw6in5iY/en/u9+F/ftPvTabhb/6K3jkkbmvjQK1ZKReITe3Ewmm+OD5p57K/YHZB9ND7mY6OQm/+AW8/vW51z38MLz6KpxzDlxxRbSOpty5MxdShX+HQtD19ubOWYbZZy2XeyzJpRaCJFZxKBSUHkRfeM0ll+RC4ciRXBi87nXwkY9AV4T+C6qm1VNLN5kkT4R+nUWq06wukcLNsFjxzbLALNcSGBs79dzAQK6lkM02p5ZmKITXhg25ELj99tw/C60eqL6bTJJJXUbSUarpEqlG8c2wcMMsPIbZLYXCmMFrr8HixbkwWLo0N4Zw4MCplkK9tTRTIRQK/x6l/y7VdJNJcoU8MW21mT1qZs+aWdrMPh6qFukMtQwEV2KWu3EX3wwLn657e2ePITz8MLz0Um7M4I47TnUf9fbCoUO577f603a1raJKrZ5quskkuUK2EDLAp919r5kNAHvM7CF3fzZgTRJh1Q4EV2vTptkDqoXrl44h9PbC5Zfnuo26uk7VsGhRbsC51Z+2q20VVdPqgfKBoVAQCHti2svAy/mvx8zsOWAQUCDIvCp1idRzvYUew8LB4d68WsopbhUVrl980y+tq1yrB3LPQ/XdZJJMkRhDMLO15I7TfDJsJRJ183WJtPpmVi442lFLra2iSq2ehQJDYSDBA8HM+oF/AD7h7q+U+f4wMAzwhjesaXN1EiW1DASHquXJJ2fX0ow5/rW2ihZq9VTTTSbJFXTaqZn1kAuDu939G+Ve4+5b3X29u6/v71/e3gIlUqodCA5VS29vbvHaokWzWxCNzvGvdnpsLbUv9FiSK1gLwcwM2AY85+53hqpDOkuUPuEW11Lo6z9+PDfQXNqCqLelEKVWUatpBXV4IbuMNgJ/COw3s2fyz93q7g8GrEk6QJQ+4ZYGEzR31lGlgeK43DCbtb5EGhNyltH3gJj8Oos0fwZUQZRaRa1Qy0wqaa3gg8oicdHKWUdRahU1W6taV1I77WUk0gSlff233npqT6FGBoCTQiuoo0GBIC2XhP35ozQDqhNVM5MqCb9HoanLSFoqSYOFce/rb5VqZlI99lhyfo9C6qgWQuZ1P+e5zP7QZUiVmrkZXaeIc19/q1RqXUHyfo9C6agWwuv6Upx+aZqjpGee+9kTQ5ybWhewKpmPBgulWpVaV/o9ao+OCoQe62HL0NDM48OZw+xmdkC8cfe1IUqTebRqKqbEz0KtK/0etUdHBUKpwdQgg0ODM48fPZjm6Prts16jgAirkamYWrkqBaE2NexoO3fCrl01/ZWODoRSl60ZmvVYARFWI9suJGkwWhaWpO07GlImAIY3pmHTJm66qbpLxCoQSpUGRHoizQinAmLi2FJWv3hlu8tKjHq3XdDK1Xiqt8WXlO076nLHHbMeFgLglNo+PZl30BD90MVDvv3x7ZVfWKX70+k5z6kF0Xz13AiKPxUWaBCxczWjxZfN5k6sm+9xIpQEAMDwLadX/Gt200173H19pdfFuoVQSfEANaiLqVXqmYoZ90HEJI2PNKPFVy5QHn44AV2IdQZAvRIdCKUqjUE8f+e18f7li5A4DyImbXyk0enHiepC3LYNjh6deTj8xn+ED32obW+vQFhAcUCkJ9Is+dx2jo6d+v7Pnhji2OPrYvkfcTvM9yk5zoOIibq5FWmkxRfr9SwVWwDtCwNQIFRtqHeIoaITPA9nDnNgIJ1fKJejRXLVq/QpOa6DiLG+uS2g0RZfbLoQ29wFVCsFQp0GU4MMrhmc9dyjA2mOjp0aqH7+ztz4g1oQs1XzKTnO+wLF5uZWpWa0+Dq6C7F0JlCEAqBU0EAws68BVwNH3f38kLU0Q3EX06MH02z43HZeGWOmBaGAyKn2U3Jc9wXq6JtbHRqdNtpRXYjl1gK0eRygEaFbCH8DfBn428B1NF25AerigEj6+EPSPiUXdNTNrYkaafFFeh1CySAwlFsL0BlhAIEDwd13mtnakDW0y5yAKBl/mDi2lP/3d1cmJiDq/ZTc6dM1I31za7FGWnyR6UIsFwBzuoA69z/i0C2ExCrXglheMosprtNc6/2UHJfpmpG5uXWYYF2IHTQG0KjIB4KZDQPDACtXrwxcTevM18VUCIg4jT/U8yk5btM14zo+EgsRnwnUSsG3rsh3GT1QzaBys7eu6BTpiTRH/w1eKWk9QGcHRK3dP9rOQlqitAXQQYPA1dLWFTFSbg3EkpIZTJ02SF3PWEBSB6KlySpuCBevMKhF6Gmn9wDvBpaZ2UvA59x9W8iaOkHpGohyi+Si3IKodywgadM1pUkqdgFF8D+SQELPMrou5PvHRWlARHkNRL1jAUmdril1SPAYQKPUZRRDldZAQLiAqHfrhiRP15QKyi0GUwDURYGQAJUCot1rIOodC9B0TQGqXAwm9VAgJFC5gOj91HaK/xNr5RqIRsYCNF0zgarqAlIYNIMCQSqugYDmdTFpLECqkqDFYFGiQJA5KnUxNRIOcRkL6PQtNCInAWsBOkHwhWm1SOrCtCgpt0iunjUQnXxDjcsWGsF0+I6gnUgL06Qlqj0oqFJAdOpYQNy20GgbLQbrCAoEach8BwV1yiK5WiX1xLOaaTFYR1IgSNPF/aAgbaFRhhaDxYICQVqqdIA6PZGesw9Tp23zrS00mDMOoDGAeFAgSFuVjkGkJ9JQtAYi6gcFJXbabMWDYUrC4PbbYWyMOQYG4NZbm1+fNIUCQYIa6h1iqKgREfWDguIybbaiRruAxsagv7/88xJZCgSJlPm6mKIUELHdQqMoBNQFlEwKBIm0Sl1Moc6B6NRpszPKjQG8kaIQUBgkkQJBOkpxF1N6Is2ZZ6YZKVkDAXBual2YAqNo587cP6sMgE5eNCiNqRgIZrYEWO7uPyl5/gJ3/2HLKhOpYKg3d/MvBMThzGGOn5lmZASOkgaiP0jdMuVaABvPKprrW74FsHXnOYxN9PCpzftnVmHfuWMdA71TDG96vg2FS0gLBoKZ/R7wReComfUAN7r70/lv/w3w9kbe3MyuBP470A181d3/vJHrSbINpgYZTA3OGqROT6TpXR6+i6nlmtAF5A5jEz3c89SZAHxq837u3LGOe546k+s2jNTWUhgYmH+WkUTWgnsZmdkzwFXu/rKZbQD+FrjF3e8zsx+4+0V1v7FZN3AA+HXgJeBp4Dp3f3a+v6O9jKQR6Ylcq2Fk5NRzHdnFNF8XEDQ8EFxoERRCAeC6DSMzLYaW0lTVlmnWXkbd7v4ygLs/ZWaXAQ+Y2Wqg0V3xNgAj7v6vAGa2HdgCzBsIIo0o7WKaGYMo6mL62RND0QyHnTvhxz+eWQvQqkFgs1zLoDgQ2hIGoKmqEVApEMbM7IzC+EG+pfBu4B+BoQX/ZmWDwKGixy8BlzR4TZGqlQsISM8afzjxk8EwAVESAJAPgVtaOwuo0EIodueOde0LBQmqUiB8BJj1a+DuY/m+/99rWVVFzGwYGAZYuXplO95SEqp0kVx6Is3Rf3eco2Ppmeda1oKYb0voW4pv/K2dClrcXVToJiruPlIoxF+lQHgVeBMwUvL8BuCJBt/7MLC66PGq/HOzuPtWYCvkxhAafE+RqpVbA5HrYmpCQFQcBIa2rQXI990bMDD+H7jO+/nUw/8Te3qAT92S67sf6J1SGCRApUD4InBLmedfyX/vtxp476eBs8zsLeSC4Frg+gauJ9JSlbqYFhygnm8gOAoLwYr67of7v5GfTdSfCwlTyyBJKgXCm9x9f+mT7r7fzNY28sbunjGzjwLfJjft9Gvunq7w10Qio7iLqdwaiJ/932Wce9fTM6+vdi1AaMFWYWuqanCVAmHpAt87rdE3d/cHgQcbvY5IaKVrIA5nDrN70wjDI+nIB0BkaGppcF0Vvr/bzD5c+qSZ/TGwpzUlSRSNZkbZO76Xx08+zt7xvYxmRkOXFGmDqfwpcrFbASdxVqmF8AngPjP7A04FwHpgEfDeVhYmtRvNjHIwc5BxH6fP+liTWsOy1LKmXPfA1AG66CJFikmf5MDUAYCmXF9EomHBQHD3fwPemV+Qdn7+6X9y90daXpnUZDQzynOTzzHNNI4z6ZM8N/kc53Juwzftg5mDdNFFt3UD0E0309lpnp18ltRUqqnhIwGo717yKu1l1AfcDJwJ7Ae2uXumHYVJbUYmR8hw6v8ax8mQYWRypOEb9biPkyr6Vcl4hgkmMIw++tRi6HTqu5e8SmMId5HrItoPXAX8ZcsrkrqMMw6AFf2v+PlG9FkfWbIzjyd9EsPoogszo9u66aKLg5mDDb+XiIRTKRDOc/cb3P2vgfcDGiGLKJ9na6n5nq/FmtQasmSZ9mncnWmmAeihZ+Y1XXQx7o2HT9x8cp+6XaRzVAqEqcIX6iqKtsW2eM7N33EW2+KGr70stYy39ryVRbaIDBlSpOihh56uU4GQJUuf9TX8XnGy/pK19K8/GboMkapVmmV0oZm9kv/agNPyjw1wd1/S0uqkamf0nMHzk8+TIUOWLF10sYhFnNFzRlOuvyy1bGZ8oDDraNqn6aKLbP5/a1JrKlxFRKKs0iyj7nYVIo1ZllrGOZzTkmmn5d4LaMt7iUj76EzlGCn+FB+n9xKR9lAgiCSBTiOTKigQatCqlcBxrUsiRKeRSRUUCFWK6vYNUa1LBGhfy0QtoKZQIFSp3PYNeO75kDfeqNYlArSvZVLt+yg4FqRAqFLp9g0QjcVYUa1LJJLUdbagSgvTJK90+waIxmKsqNYlIp0nSCCY2TVmljazrJmtD1FDreZs3+DTkViMFdW6JGIGBuDEibl/tKOpFAnVZfQj4HeBvw70/jWL6mKsqNYlOb1vnoSRCDTE1T8uVQgSCO7+HIB12MndUV2MFdW6km4wNchuXoBt2+BDCT0+s9azFuod9NWZDk2hQWVpiNZAVNCX8LGcalomxSFw/DgUPih2d8OKFbmvKw36VtsCUnAsqGWBYGY7gBVlvnWbu99fw3WGgWGAlatXNqk6adRoZpSRyRFe47WZjfQm0RoIqUPxzJ9f/jIXBADT081/L3WdLahlgeDum5t0na3AVoChi4ca39xfGlZYDDfJ5MxBPBNM0EffzEE5CgRpK60vaAp1GUnNCovhis9fMIxJn+Q0O01rIKT9tL6gKYIEgpm9F/gfwHLgn8zsGXf/jRC1SO0Ki+EKZyEUFM5F0BoIaYvSsYdf/jL3dfHYg9Qk1Cyj+4D7Qry3NK7P+pj0SXroYYKJmZaCYVoDIY3p7j41duCeWysBcwd9b78dDh06NQCdzX8w6bCZi1GjLiOp2ZrUmtyGetbFIl/EFFNkybLYFnNGzxkaP5DaFM/8Ke72Waj/f2wsd/MvHoA2y4WI1E2BIDWbtRiOcZbYEk03lfpp0DcyFAhSFy2Gk0gpbh0s1NUkC1IgiEjn6+nJ/XN6Gn7lV+COO8LW06EUCCLSWq1aI1A8AF3gPrtVoPUJNVEgiLTYf73jj/jsSOgqAmrFGoH5uoJKb/Ran1ATBUKMaZ+h8LZcfDH379oVuozoqvcTvD7dt4QCIaZ01rJ0BH2Cj5QIbNQurVB81rKZ0W3dM/sMiYiUoxZCTNV61rK6l0REgRBThe0luumeeW6+fYbUvSQttdAZBK3uGtL5BzVRIMRUYXsJnJlN6ObbZ6i4ewnIhYijbaylORYaAL7llnDvLXMoEGKqlrOWa+1eEmkafYKPFAVCjFW7vUQt3UsSU6EWcOkTfKQoEKSm7iWJKU3/FBQIwqnupZ9M/YRX/VUMo4/KrQPNTBKJlyDrEMzs82b2vJn90MzuM7OlIeqQ2aaZ5jQ7jcW2GAwOTB1gNDNa9rUz5yr75KyZSfO9XkSiL9TCtIeA8939AuAA0OKpBlJJrQvZtPBNJH6CBIK7/7O7Z/IPnwBWhahDThn3cbpKfh0WmmlU6+uT7k971Dsr0ReFrSs+CHwzdBFJ12d9ZMnOem6hmUa1vj7J1l+yNnQJlQ0M5A6VKf2j6Z+J0rKPLWa2A1hR5lu3ufv9+dfcBmSAuxe4zjAwDLBy9coWVCpQ+0wjzUyKGU3/FFoYCO6+eaHvm9mNwNXAFe7zn4zt7luBrQBDFw/pBO0WqWUhWz2vF5HoC9KxaWZXAp8Bfs3dXwtRg8xV6znJOldZJF5CjXR9GegFHjIzgCfc/eZAtYiI6LhNAgWCu58Z4n1FJMJC35C1WlsrlUUkInRDDi4K005FRCQCFAgiIgLEoMvIM44fcZgIXckCesFWGJay0JWIiMyr8wPhiLNsyTKWvmEp+RlLkeLuHP/5cUaPjGKrolefiOTpsJ7ODwQmiGwYAJgZS9+wlNFj2gU0yXrfPAlf+x5s2hS6lOgKfUNOyNTShXR+IEBkw6Ag6vVJaw2mBtm99Dhbdw0xrDyYn27IwWlQuQlu/tDNvHnFm1l/wfrQpYiI1C0WLYTQbvjADdz0Jzfx4Rs/HLoUEQkt9AK7BiQqEHo3/Cp27Nic5335ciaeeqzu675r07t48YUXGylNROKigxfYJSoQ7NgxfPnczdjKhYSISNJoDEFERAAFgoiI5CkQREQEUCA0xQeu/wCXbbyMH//LjzlrzVncte2u0CWJSCgdfD51ogaVffnyeWcZNeKu/60AEJG8iE8tXUioIzT/DNgCZIGjwI3u/tNWv28jU0tFROIuVJfR5939And/G/AA8J8D1SEiInlBAsHdXyl6+DrAQ9Qh0i5LBuDotteHLkNkQcEGlc3sv5nZIeAPUAtBYu6yNUOhSxCpqGWBYGY7zOxHZf5sAXD329x9NXA38NEFrjNsZrvNbPcvjv2iVeWKiCReywaV3X1zlS+9G3gQ+Nw819kKbAUYunhIXUsiIi0SpMvIzM4qergFeD5EHc3y0qGXuOqKq7j4/ItZv249X/nSV0KXJCJSs1DrEP7czM4mN+30ReDmdr2xOxSfV1P6uB7dqW5u//ztXPT2ixgbG+Nd73gXl2++nHPPO7exC4uItFGQQHD394V4369t7WbshPGxT2Ywy4XBl76QYqDf+eDwdN3XXblyJStXrgRgYGCAs885m58e/qkCQUQ6SmK2rnCHsRPGvdu7+dIXUjNhcO/2XEh4k0YnXnzhRfY9s493XPKO5lxQRKRNErN1hRl87JMZAO7d3s2927sBuOba6ZkWQ6NOnDjB9ddcz1/c+RcsWbKk8QuKiLRRYloIMDsUCpoVBlNTU1z//uv5/et/ny2/u6XxC4qItFmiAqHQTVSs0H3U2HWdj/zxRzj73LP52Cc/1tjFpCGjmVH2ju/l8ZOPs3d8L6OZ0dAliXSMxARC8ZjBNddO89iTE1xz7fSsMYV6fX/X97nn6/fw3Ue/y6Vvv5RL334p33rwW80rXqoymhnlwNQBJn2SFCkmfZIDUwcUCiJVStQYwkC/zxozKHQfDfR7Q91G73zXO3l1+tUmVSr1Opg5SBdddFtufKibbvDc88tSc8/SDuFPe1L82VSm8gtFAkhMIAB8cHh61rqDQig0YwxBwhv3cVIlv9JddDHu44Eqmm3Lxo3czy4YSUzDXDpM4n4zS2/+CoP46LM+smRnPZclS5/1BapIpLMkLhAkvtak1pAly7RP4+5M+zRZsqxJrQldmkhHSFSXkcRbYZzgYOYg4z5On/WxJrUmMuMHIlGnQJBYWZZapgAQqZO6jEREBFALoSnGx8d5z7vfw8TEBNOZaX7nfb/DZ//LZ0OXJSJSEwVCE/T29vLgjgfp7+9namqKzZs2854r38OGSzeELk1EpGqJC4RDU4fYN7GPsewYA10DXNh7Iat7Vjd0TTOjv78fyO1pNDU1hWk+q4h0mESNIRyaOsSuk7t4zV+j13p5zV9j18ldHJo61PC1p6enufTtl7J2xVou33y5tr8WkY4TNBDM7NNm5mbWlmkh+yb20WVd9FgPZkaP9dBlXeyb2Nfwtbu7u3li7xMcOHiAPU/vIf2jdBMqFhFpn2CBYGargfcAB9v1nmPZsTlbG6RIcSJ7omnvsXTpUja9exMPffuhpl1TRKQdQrYQvgB8BmjSWWWVDXQNkGH2xmIZMvR39Td03WPHjnH8+HEATp48ySM7HuHss89u6JoSX1vv+FnoEkTKCjKobGZbgMPuvq/S4KuZDQPDACtXr2zofS/svZBdJ3cxxRQpUmTIkPUsF/Zd2NB1j7x8hOE/GmZ6eppsNsv7rnkfV119VUPXlJhauhT4RegqRMpqWSCY2Q5gRZlv3QbcSq67qCJ33wpsBRi6eKih1sTqntVsZCP7JvZxInuC/q5+LuxrfJbRugvW8f0932/oGiIiobUsENx9c7nnzWwd8Bag0DpYBew1sw3ufqRV9RSs7lndcACIiMRR27uM3H0/8MbCYzN7AVjv7jrWSkQkoEStQxARkfkFX6ns7mubcI1Irwz2Rg5sFhFpk85vIfTC8Z8fj+xN1905/vPj0Bu6EhGRhQVvITTKVhijR0YZPRbhIYjeXJ0iIlHW+YGQMmyVbrYiIo3q/C4jERFpCgWCiIgACgSRtju67fWhSxApy6I6O6ccMzsGvBi6jnksAyI8sh2Efiaz6ecxl34mc7XiZ/Jmd19e6UUdFQhRZma73X196DqiRD+T2fTzmEs/k7lC/kzUZSQiIoACQURE8hQIzbM1dAERpJ/JbPp5zKWfyVzBfiYaQxAREUAtBBERyVMgtICZfdrM3MyWha4lJDP7vJk9b2Y/NLP7zGxp6JpCMbMrzexfzGzEzP5T6HpCMrPVZvaomT1rZmkz+3jomqLCzLrN7Adm9kCI91cgNJmZrSZ3POjB0LVEwEPA+e5+AXAAuCVwPUGYWTfwFeAq4DzgOjM7L2xVQWWAT7v7ecClwJ8k/OdR7OPAc6HeXIHQfF8APgMkfnDG3f/Z3TP5h0+QOy41iTYAI+7+r+4+CWwHtgSuKRh3f9nd9+a/HiN3AxwMW1V4ZrYK+E3gq6FqUCA0kZltAQ67+77QtUTQB4Fvhi4ikEHgUNHjl9ANEAAzWwtcBDwZtpJI+CK5D5PZUAV0/PbX7WZmO4AVZb51G3Arue6ixFjo5+Hu9+dfcxu5boK721mbRJuZ9QP/AHzC3V8JXU9IZnY1cNTd95jZu0PVoUCokbtvLve8ma0D3gLsyx/nuQrYa2Yb3P1IG0tsq/l+HgVmdiNwNXCFJ3eO82FgddHjVfnnEsvMesiFwd3u/o3Q9UTARuC3zezfA33AEjP7urvf0M4itA6hRczsBWC9uyd24y4zuxK4E/g1dz8Wup5QzCxFblD9CnJB8DRwvbungxYWiOU+Md0F/NzdPxG6nqjJtxD+o7tf3e731hiCtNKXgQHgITN7xsz+V+iCQsgPrH8U+Da5AdS/T2oY5G0E/hC4PP978Uz+k7EEphaCiIgAaiGIiEieAkFERAAFgoiI5CkQREQEUCCIiEieAkGkCmY2nZ8e+SMzu9fMFuefX2Fm283sJ2a2x8weNLO35r/3LTM7HmrnSpFaKRBEqnPS3d/m7ucDk8DN+QVW9wHfcfcz3P1icju6vin/dz5Pbr69SEdQIIjU7jHgTOAyYMrdZxbcufs+d38s//XDwFiYEkVqp0AQqUF+G4qrgP3A+cCesBWJNI8CQaQ6p5nZM8BucocfbQtcj0jTabdTkeqcdPe3FT9hZmng/YHqEWk6tRBE6vcI0Gtmw4UnzOwCM/vVgDWJ1E2BIFKn/PkO7wU256edpoE7gCMAZvakc9HHAAAARklEQVQYcC9whZm9ZGa/Ea5akcq026mIiABqIYiISJ4CQUREAAWCiIjkKRBERARQIIiISJ4CQUREAAWCiIjkKRBERASA/w8RTPdOXpzHOgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff3a686bcd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_decision_regions(X_test_pca, y_test, classifier=lr)\n",
    "plt.xlabel('PC1')\n",
    "plt.ylabel('PC2')\n",
    "plt.legend(loc='lower left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "啊哈！测试集仅有一个样本被错误分类，效果很棒。\n",
    "\n",
    "怎样用sklearn的PCA得到每个主成分(不是特征)的方差解释率呢？很简单，初始化PCA时，n_components设置为None，然后通过explained_variance_ratio_属性得到每个主成分的方差解释率："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "pca = PCA(n_components=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_pca = pca.fit_transform(X_train_std)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.37329648,  0.18818926,  0.10896791,  0.07724389,  0.06478595,\n",
       "        0.04592014,  0.03986936,  0.02521914,  0.02258181,  0.01830924,\n",
       "        0.01635336,  0.01284271,  0.00642076])"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pca.explained_variance_ratio_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此时n_components=None, 我们并没有做降维。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## LDA进行监督数据压缩\n",
    "\n",
    "LDA(linear discriminant analysis, 线性判别分析)是另一种用于特征抽取的技术，它可以提高计算效率，对于非正则模型也能减小过拟合。\n",
    "\n",
    "虽然LDA的很多概念和PCA很像，但他俩的目标不同，PCA目标是找到正交的主成分同时保持数据集的最大方差，LDA的目标是为每个类单独优化，得到各个类的最优特征子集。PCA和LDA都是线性转换技术，用于数据压缩，前者是无监算法，后者是监督算法。看到监督两个字，可能你会认为对于分类任务，LDA要比PCA效果更好，但实际却不是这样，在某些分类任务情境下，用PCA预处理数据得到的结果要比LDA号，比如，如果每个类含有的样本比较少。\n",
    "\n",
    "下图画出了对于二分类问题，LDA的一些概念：\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576ce4747d450.png)\n",
    "\n",
    "x轴的线性判别(LD 1),很好地将两个正态分布的类别数据分离。虽然y轴的线性判别(LD 2)捕捉了数据集中的大量方差，但却不是一个好的线性判别，因为它没有捕捉任何与类别判别相关的信息。\n",
    "\n",
    "\n",
    "\n",
    "LDA 的一个假设是数据服从正态分布。同时，我们也假设各个类含有相同的协方差矩阵，每个特征都统计独立。即使真实数据可能不服从上面的某几个假设，但LDA依然具有很好的表现。\n",
    "\n",
    "\n",
    "在学习LDA内部原理前，我们先将它的几大步骤列出来：\n",
    "* 1. 将d维度原始数据进行标准化.\n",
    "* 2. 对每一个类，计算d维度的平均向量.\n",
    "* 3. 构建类间(between-class)散点矩阵$S_{B}$和类内(within-class)散点矩阵$S_{W}$.\n",
    "* 4. 计算矩阵$S_{W}^{-1}S_{B}$的特征向量和特征值.\n",
    "* 5. 选择值最大的前k个特征值对应的特征向量，构建d*d维度的转换矩阵$W$,每一个特征向量是$W$的一列.\n",
    "* 6. 使用矩阵$W$将原始数据集映射到新的特征子空间.\n",
    "\n",
    "**Note** 我们在应用LDA时做出的假设是：特征服从正态分布并且彼此独立，每个类的协方差矩阵都相同。现实中的数据当然不可能真的全部服从这些假设，但是不用担心，即使某一个甚至多个假设不成立，LDA也有不俗的表现.(R.O.Duda, P.E. Hart, and D.G.Stork. *Pattern Classification*. 2nd.Edition. New York, 2001)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 计算散点矩阵\n",
    "\n",
    "数据标准化前面已经说过了，这里就不再讲了，我们说一下如何计算平均向量，然后用这些平均向量分别构建类内散点矩阵和类间散点矩阵。\n",
    "\n",
    "每一个平均向量$m_{i}$存储了类别i的样本的平均特征值$u_{m}$:\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576cf81b58b4d.png)\n",
    "\n",
    "Wine数据集有三个类，每一个的平均向量：\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576cf85702d4d.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.set_printoptions(precision=4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean_vecs = []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MV 1: [ 0.9259 -0.3091  0.2592 -0.7989  0.3039  0.9608  1.0515 -0.6306  0.5354\n",
      "  0.2209  0.4855  0.798   1.2017] \n",
      "\n",
      "MV 2: [-0.8727 -0.3854 -0.4437  0.2481 -0.2409 -0.1059  0.0187 -0.0164  0.1095\n",
      " -0.8796  0.4392  0.2776 -0.7016] \n",
      "\n",
      "MV 3: [ 0.1637  0.8929  0.3249  0.5658 -0.01   -0.9499 -1.228   0.7436 -0.7652\n",
      "  0.979  -1.1698 -1.3007 -0.3912] \n",
      "\n"
     ]
    }
   ],
   "source": [
    "for label in range(1, 4):\n",
    "    mean_vecs.append(np.mean(X_train_std[y_train == label], axis=0))\n",
    "    print('MV {}: {} \\n'.format(label, mean_vecs[label-1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "有了平均向量，我们就可以计算类内散点矩阵$S_{W}$:\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576cf97a62714.png)\n",
    "\n",
    "$S_{W}$就是每个类的散点矩阵$S_{i}$总和。\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576cf998c8d6b.png)\n",
    "\n",
    "代码："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "d = 13 # 特征维度"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "S_W = np.zeros((d, d))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Within-class scatter matrix: 13x13\n"
     ]
    }
   ],
   "source": [
    "for label, mv in zip(range(1, 4), mean_vecs):\n",
    "    class_scatter = np.zeros((d, d))\n",
    "    for row in X[y==label]:\n",
    "        row, mv = row.reshape(d, 1), mv.reshape(d, 1)\n",
    "        class_scatter += (row-mv).dot((row-mv).T)\n",
    "    S_W += class_scatter\n",
    "\n",
    "print('Within-class scatter matrix: {}x{}'.format(S_W.shape[0], S_W.shape[1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当我们计算散点矩阵时，我们做出的假设是训练集中的类别时均匀分布的。实际情况往往并不是这样，比如我们将Wine训练集各个类别个数打印出来："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Class label distribution: [40 49 35]\n"
     ]
    }
   ],
   "source": [
    "print('Class label distribution: {}'.format(np.bincount(y_train)[1:]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "所以在得到每个类别的散点矩阵$S_{i}$后，我们要将其缩放，然后再相加得到$S_{W}$。如果我们将散点矩阵$S_{i}$除以各个类别内样本数$N_{i}$,我们实际上是在计算协方差矩阵$\\Sigma_{i}$. **协方差矩阵是散点矩阵的归一化结果**：\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576cfbe871c14.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Scaled within-class scatter matrix: 13x13\n"
     ]
    }
   ],
   "source": [
    "d = 13 # 特征维度\n",
    "S_W = np.zeros((d, d))\n",
    "\n",
    "for label, mv in zip(range(1, 4), mean_vecs):\n",
    "    class_scatter = np.cov(X_train_std[y_train==label].T)\n",
    "    S_W += class_scatter\n",
    "\n",
    "print('Scaled within-class scatter matrix: {}x{}'.format(S_W.shape[0], S_W.shape[1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "得到缩放后的类内散点矩阵后，我们接下来计算类间散点矩阵$S_{B}$:\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576cfcd0b45f6.png)\n",
    "\n",
    "其中，$m$是整个训练集所有样本的特征平均值.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean_overall = np.mean(X_train_std, axis=0)\n",
    "d = 13\n",
    "S_B = np.zeros((d, d))\n",
    "\n",
    "for i, mean_vec in enumerate(mean_vecs):\n",
    "    n = X[y==i+1, :].shape[0]\n",
    "    mean_vec = mean_vec.reshape(d, 1)\n",
    "    mean_overall = mean_overall.reshape(d, 1)\n",
    "    S_B += n*(mean_vec - mean_overall).dot((mean_vec - mean_overall).T)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Between-class scatter matrix: 13x13\n"
     ]
    }
   ],
   "source": [
    "print(\"Between-class scatter matrix: {}x{}\".format(S_B.shape[0], S_B.shape[1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 为特征子空间选择线性判别式\n",
    "\n",
    "\n",
    "剩下的LDA步骤和PCA很像了。不同点是PCA分解协方差矩阵得到特征值和特征向量，LDA分解$S_{W}^{-1}S_{B}$得到特征值和特征向量：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "eigen_vals, eigen_vecs = np.linalg.eig(np.linalg.inv(S_W).dot(S_B))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "得到特征值后，对其降序排序："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "eigen_pairs = [(np.abs(eigen_vals[i]), eigen_vecs[:, i])\n",
    "              for i in range(len(eigen_vals))]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "eigen_pairs = sorted(eigen_pairs, key=lambda k: k[0], reverse=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Eigenvalues in decreasing order: \n",
      "\n",
      "643.015384346\n",
      "225.086981854\n",
      "8.74464511186e-14\n",
      "7.37188088351e-14\n",
      "6.047296304e-14\n",
      "6.047296304e-14\n",
      "5.87718844433e-14\n",
      "2.84411108606e-14\n",
      "2.84411108606e-14\n",
      "2.36106427714e-14\n",
      "2.36106427714e-14\n",
      "7.02000577228e-15\n",
      "1.05565969043e-16\n"
     ]
    }
   ],
   "source": [
    "print(\"Eigenvalues in decreasing order: \\n\")\n",
    "for eigen_val in eigen_pairs:\n",
    "    print(eigen_val[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以看到只有两个特征值不为0，其余11个特征值实质是0，这里只不过由于计算机浮点表示才不为0。极端情况是特征向量共线，此时协方差矩阵秩为1，只有一个非0特征值。\n",
    "\n",
    "\n",
    "为了度量线性判别式(特征向量)捕捉到了多少的类判别信息，我们画出类似方差解释率的线性判别图："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "tot = sum(eigen_vals.real)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "discr = [(i/tot) for i in sorted(eigen_vals.real, reverse=True)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "cum_discr = np.cumsum(discr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEKCAYAAAD5MJl4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3Xt8FPW9//HXhyAG5VIR5IABAxougZCIAUG8gPWCNxARFUEKKh5tUby15fzqUWr9tfbUHuqtUrQWL1RRqpYq1YKIWgQhICg3CwJK0GrEKmhFLvmcP3YSl0yymYRsloX38/HYx+7Mfmfms5PLZ+c7M5+vuTsiIiLxGqQ6ABER2fcoOYiISIiSg4iIhCg5iIhIiJKDiIiEKDmIiEiIkoOIiIQoOYiISIiSg4iIhDRMdQA11bJlS8/Ozk51GCIiaWXJkiWfunurqO3TLjlkZ2dTVFSU6jBERNKKmb1fk/bqVhIRkRAlBxERCVFyEBGRECUHEREJUXIQEZEQJQcREQlRchARkRAlBxERCVFyEBGRECUHEREJUXIQEZEQJQcREQlRchARkZCkJQcze9jMPjGzFVW8b2Z2j5mtM7O3zaxnsmIREZGaSeaRw1RgYIL3zwJygsdVwANJjEVERGogaeM5uPtrZpadoMlg4FF3d2ChmX3HzNq4+0fJiikZ/vjmB/x52eZUhyEiaSy3bTNuO69bqsPYQyrPORwJbIqbLg7mhZjZVWZWZGZFJSUl9RJcVH9etplVH21NdRgiInUqLUaCc/cpwBSAwsJCT3E4IbltmjH9P/umOgwRkTqTyiOHzUC7uOmsYJ6IiKRYKpPDTGBUcNVSH+CLdDvfICKyv0pat5KZPQH0B1qaWTFwG3AQgLtPBmYBZwPrgH8DY5IVi4iI1Ewyr1YaXs37DvwgWdsXEZHa0x3SIiISouQgIiIhSg4iIhKi5CAiIiFKDiIiEqLkICIiIUoOIiISouQgIiIhSg4iIhKi5CAiIiFKDiIiEqLkICIiIUoOIiISouQgIiIhSg4iIhKi5CAiIiFKDiIiEqLkICIiIUoOIiISouQgIiIhSg4iIhKi5CAiIiFKDiIiEqLkICIiIUoOIiISouQgIiIhSg4iIhKS1ORgZgPN7F0zW2dmEyp5v72ZvWJmb5nZ22Z2djLjERGRaJKWHMwsA7gfOAvIBYabWW6FZrcAT7n7scAlwG+TFY+IiESXzCOH3sA6d1/v7juAJ4HBFdo40Cx43Rz4MInxiIhIRA2TuO4jgU1x08XA8RXaTAT+ZmbXAocCpyUxHhERiajaIwczO8jMrjOzGcHjWjM7qI62PxyY6u5ZwNnAY2YWisnMrjKzIjMrKikpqaNNi4hIVaJ0Kz0AHEfsfMBvgZ7BvOpsBtrFTWcF8+JdATwF4O4LgEygZcUVufsUdy9098JWrVpF2LSIiOyNKN1Kvdw9P256rpktj7DcYiDHzDoQSwqXAJdWaPMB8F1gqpl1JZYcdGggIpJiUY4cdpvZ0WUTZtYR2F3dQu6+CxgHvASsJnZV0kozu93MBgXNbgLGBsnmCWC0u3tNP4SIiNStKEcOPwReMbP1gAFHAWOirNzdZwGzKsy7Ne71KqBf5GhFRKReVJsc3P1lM8sBOgez3nX3b5IbloiIpFKVycHMTnX3uWZ2QYW3jjEz3P2ZJMcmIiIpkujI4RRgLnBeJe85oOQgIrKfqjI5uPttwcvb3X1D/HvBFUgiIrKfinJC+k/E7m2IN4PYvQ/7nZ/+ZSWrPtwauf2qj7aS26ZZ9Q1FRNJIonMOXYBuQPMK5x2aEbsfIe1Mmv2Patu89cHnlGyr/nx71mGNAcht04zBBUfudWwiIvuSREcOnYFzge+w53mHbcDYZAaVSqd0inYH9g2nd0pyJCIiqZPonMOfgT+bWd+gtIWIiBwgopxzeMvMfkCsi6m8O8ndL09aVCIiklJRymc8BvwHcCbwKrECetuSGZSIiKRWlORwjLv/N/CVuz8CnEN4XAYREdmPREkOO4Pnz82sO7ER245IXkgiIpJqUc45TDGzw4iN9zwTaAL8d1KjEhGRlEqYHIJR2ba6+7+A14CO9RKViIikVMJuJXcvBX5UT7GIiMg+Iso5hzlmdrOZtTOzFmWPpEcmIiIpE+Wcw8XB8w/i5jnqYhIR2W9FGexHFVhFRA4wUbqVRETkAKPkICIiIUoOIiISkmg8h/bBy93uvrme4hERkX1AohPSjxC7Kukz4ML6CUdERPYFiZLDxOC5+mHRRERkv5IoOYwOnj8HFiY/FBER2VckGgluDICZZdRfOCIisi+IcrXSWjP7lZnlJj0aERHZJ0RJDvnAP4CHzGyhmV1lZs2irNzMBprZu2a2zswmVNHmIjNbZWYrzeyPNYhdRESSpNrk4O7b3P1Bdz8B+DFwG/CRmT1iZsdUtVzQHXU/cBaQCwyvePRhZjnAfwH93L0bcH3tP4qIiNSVapODmWWY2SAzexb4DfBrYkX3/gLMSrBob2Cdu6939x3Ak8DgCm3GAvcH40Xg7p/U4jOIiEgdi1KVdS3wCvArd38jbv4MMzs5wXJHApviposJjz3dCcDM5gMZwER3fzFCTCIikkRRksMod/97/Awz6+fu8939ujrYfg7QH8gCXjOzPHf/vML2rgKuAmjfvn3FdYiISB2LckL6nkrm3Rthuc1Au7jprGBevGJgprvvdPcNxE5851RckbtPcfdCdy9s1apVhE2LiMjeSFRbqS9wAtDKzG6Me6sZsS6g6iwGcsysA7GkcAlwaYU2zwHDgT+YWUti3Uzro4cvsvd27txJcXEx27dvT3UoInstMzOTrKwsDjrooL1aT6JupUZAk6BN07j5W4lQa8ndd5nZOOAlYsnkYXdfaWa3A0XuPjN47wwzWwXsBn7o7ltq91FEaqe4uJimTZuSnZ2NmaU6HJFac3e2bNlCcXExHTrs3Thtie6QfhV41cymuvv7tVm5u8+iwhVN7n5r3GsHbgweIimxfft2JQbZL5gZhx9+OCUlJXu9rkTdSr9x9+uB+8zMK77v7oP2eusi+wglBtlf1NXvcqIT0o8Fz3cRu7eh4kNE0tDGjRvp3r17tW3++MdvCxYUFRVx3XV7e3EizJs3j9GjRzN16lQmTpxYaZsmTZoA8OGHH3LhhXs/WsDMmTO58847a7TM2Wefzeeff159w0qMHj2aGTNmhObH78OpU6cybtw4ACZPnsyjjz5aPv/DDz+sdhvZ2dl7PCdDom6lJcHzq0nbuojsk8qSw6WXxq4hKSwspLCwsF5jaNu2baX/ZGti165dDBo0iEGDatbRMWtWovt7a6eqfXj11VeXv546dSrdu3enbdu2db79mqryyMHM3jGzt6t61GeQIvu7Rx99lB49epCfn89ll10GhL+Bln2jnjdvHqeccgqDBw+mY8eOTJgwgWnTptG7d2/y8vJ47733Ei4fb+PGjZx00kn07NmTnj178sYbsftcJ0yYwOuvv05BQQGTJk1i3rx5nHvuuZSWlpKdnb3Ht+qcnBw+/vhjSkpKGDp0KL169aJXr17Mnz8/tL1GjRrRvHlzGjduXB7Phg0b6Nu3L3l5edxyyy17xFZ2hLNy5Up69+5NQUEBPXr0YO3atQn329VXX83xxx/Pj370oz2+pY8ePZprrrmGPn360LFjR+bNm8fll19O165dGT16dPm2s7Oz+fTTT9m4cSNdu3Zl7NixdOvWjTPOOIOvv/4agAcffJBevXqRn5/P0KFD+fe//12+/Jw5cygsLKRTp048//zz5T+3c889N7RPJk6cyF133cWMGTMoKipixIgRFBQU8MILL3D++eeXt5s9ezZDhgwBoOyS/mRe2p/oaqXwpxDZz/30LytZ9eHWOl1nbttm3HZetyrfX7lyJXfccQdvvPEGLVu25LPPPqt2ncuXL2f16tW0aNGCjh07cuWVV7Jo0SLuvvtu7r33Xn7zm99Eiu2II45g9uzZZGZmsnbtWoYPH05RURF33nknd9111x7/2AAaNGjA4MGDefbZZxkzZgxvvvkmRx11FK1bt+bSSy/lhhtu4MQTT+SDDz7gzDPPZPXq1Xts74QTTuCEE07YY9748eO55pprGDVqFPfff3+lcU6ePJnx48czYsQIduzYwe7duxPut+LiYt544w0yMjKYOnXqHuv617/+xYIFC5g5cyaDBg1i/vz5PPTQQ/Tq1Ytly5ZRUFCwR/u1a9fyxBNP8OCDD3LRRRfxpz/9iZEjR3LBBRcwduxYAG655RZ+//vfc+211wKxxLZo0SLee+89BgwYwLp166r9WVx44YXcd9993HXXXRQWFuLu3HTTTZSUlNCqVSv+8Ic/cPnllwOwePHiPZ6TocojB3d/P9EjaRGJHGDmzp3LsGHDaNmyJQAtWrSodplevXrRpk0bDj74YI4++mjOOOMMAPLy8ti4cWPkbe/cuZOxY8eSl5fHsGHDWLVqVbXLXHzxxUyfPh2AJ598kosvvhiIfVseN24cBQUFDBo0iK1bt/Lll19Wu7758+czfPhwgPJv/xX17duXn//85/zyl7/k/fffp3Hjxgn327Bhw8jIqPx2rPPOOw8zIy8vj9atW5OXl0eDBg3o1q1bpfuuQ4cO5QnjuOOOK2+zYsUKTjrpJPLy8pg2bRorV64sX+aiiy6iQYMG5OTk0LFjR9asWVPtfqjIzLjssst4/PHH+fzzz1mwYAFnnXVWjddTW4muVvq7u59oZtuIjSVt8c/uHqlst0g6SfQNv741bNiQ0tJSAEpLS9mxY0f5ewcffHD56wYNGpRPN2jQgF27dlW7fJlJkybRunVrli9fTmlpKZmZmdXG1bdvX9atW0dJSQnPPfdceVdQaWkpCxcujLSOiqq7wubSSy/l+OOP54UXXuDss8/md7/7XcL2hx56aJXvxe+rivuxbN9V1h4gIyOjvFtp9OjRPPfcc+Tn5zN16tTyo6vKPk9tryAaM2YM5513HpmZmQwbNoyGDaNUPKobiY4cTgyem7p7s4rP9RahyH7u1FNP5emnn2bLltj9n2XdI9nZ2SxZsgSIXXGzc+fOGq03yvJffPEFbdq0oUGDBjz22GPs3r0bgKZNm7Jt27ZK12tmDBkyhBtvvJGuXbty+OGHA3DGGWdw773fVtZZtmxZpDj79evHk08+CcC0adMqbbN+/Xo6duzIddddx+DBg3n77ber3G/1Zdu2bbRp04adO3eG4n766acpLS3lvffeY/369XTu3DnSOivu97Zt29K2bVvuuOMOxowZU6fxVydKbSXMrKeZXWdm15rZsckOSuRA0q1bN37yk59wyimnkJ+fz403xu4JHTt2LK+++ir5+fksWLAg4bfhykRZ/vvf/z6PPPII+fn5rFmzprxNjx49yMjIID8/n0mTJoWWu/jii3n88cfLu5QA7rnnHoqKiujRowe5ublMnjw5Upx33303999/P3l5eWzeXLH8WsxTTz1F9+7dKSgoYMWKFYwaNarK/VZffvazn3H88cfTr18/unTpssd77du3p3fv3px11llMnjw58tFU2cn0goKC8iOUESNG0K5dO7p27VrnnyERi92knKCB2a3AMOCZYNb5wNPufkeSY6tUYWGhFxUV1WrZSbP/UWdx3HB6pzpbl6TW6tWr6/0PTySqcePGceyxx3LFFVdEXqay32kzW+Luka9HjtKBNQLId/ftwQbuBJYBKUkOIiIHiuOOO45DDz2UX/+6/u87jpIcPgQygbKSlQcTLr0tIiJ1rOycUSokulrpXmJXJ30BrDSz2cH06cCi+glPRERSIdGRQ1nH/hLg2bj585IWjYiI7BMS1VZ6pD4DERGRfUe15xzMLAf4BZBL7NwDAO7eMYlxiYhICkW5z+EPwAPALmAA8CjweDKDEjnQVKw3VJ34Im5RSlLfeuutzJkzJ+F6aqOsQF1l8+OfK4ovCnjllVdGKttRnZruw/hS2TWVaL/Fl/uurPz4smXLIlV9nThxIlOnTmX06NF73H1dX6JcrdTY3V82MwtqKk00syXArdUtKJKO6vJ+GIh2T0xZNdTaiFKS+vbbb6/1+pPtoYce2qvld+3aRcOGDWu8D+NLZdelyv7xx5cfX7ZsGUVFRZx99tlJ2X5diXLk8I2ZNQDWmtk4MxtCbGxpEakj8eW4+/fvz4UXXkiXLl0YMWIEZTeqvvjii3Tp0oWePXvyzDPPlC9bVpL6iy++4Kijjiqvp/TVV1/Rrl07du7cucc39arWU1Y6ukz37t3Li8ydf/75HHfccXTr1o0pU6ZU+3kqlpR2d8aNG0fnzp057bTT+OSTT8rb9u/fn6KiInbv3s3o0aPp3r07eXl55Xdmr1u3jtNOO438/Hx69uzJe++9x7x58zjppJMYNGgQubm5oX0YpaR5/Oft378/P/7xj+nduzedOnXi9ddfB6ouaQ6wdetWzjnnHDp37szVV19dvt8rO5oqKz++Y8cObr31VqZPn05BQQHTp08nJyenfFjP0tJSjjnmGEpKSmjSpAmNGzemefPmNGrUqNp9XteiHDmMBw4BrgN+BpwKfC+ZQYkcyN566y1WrlxJ27Zt6devH/Pnz6ewsJCxY8cyd+5cjjnmmD3KVpRp3rw5BQUFvPrqqwwYMIDnn3+eM888k4MOOqi8zfbt26tdT2UefvhhWrRowddff02vXr0YOnRoeU2lylQsKf3ss8/y7rvvsmrVKj7++GNyc3PLy0+XWbZsGZs3b2bFihUA5V0zI0aMYMKECQwZMoTt27dTWlrKpk2bWLp0KStWrKBDhw6h7dempPmuXbtYtGgRs2bN4qc//Slz5sypsqQ5wKJFi1i1ahVHHXUUAwcO5Jlnnql25LpGjRpx++23U1RUxH333QfAmjVrmDZtGtdffz1z5swhPz+fVq1acfPNNwNE/hnVtWqPHNx9sbt/6e7F7j7G3S9w94X1EZzIgah3795kZWXRoEEDCgoK2LhxI2vWrKFDhw7k5ORgZowcObLSZasqp10m6noquueee8jPz6dPnz5s2rSpfLCdqF577TWGDx9ORkYGbdu25dRTTw216dixI+vXr+faa6/lxRdfpFmzZmzbto3NmzeXD3KTmZnJIYccAsT2U2WJAWpX0vyCCy4A9izLnaikee/evenYsSMZGRkMHz6cv//97zXaJ2Uuv/zy8nMfDz/8cL0X2KtKtcnBzDqZ2YNm9jczm1v2qI/gRA5EFUtEV1ZGuiqDBg3ixRdf5LPPPmPJkiWV/hOuSnyJb4gdZUCsm2bOnDksWLCA5cuXc+yxx5a/V5cOO+wwli9fTv/+/Zk8eTJXXnllwvZRynJD1SXNq1omfp/HlzQvKirao+x5XZXlbteuHa1bt2bu3LksWrSoXsdsSCTKOYengaXALcAP4x4iUk+6dOnCxo0by/vLn3jiiUrbNWnShF69ejF+/HjOPffc0IA3idaTnZ3N0qVLAVi6dCkbNmwAYmW9DzvsMA455BDWrFnDwoU17zg4+eSTmT59Ort37+ajjz7ilVdeCbX59NNPKS0tZejQodxxxx0sXbqUpk2bkpWVxXPPPQfAN998s8dwnMlWVUlziHUrbdiwgdLSUqZPn86JJ54YaZ2VlUO/8sorGTlyZMJBiupblOSwy90fcPdF7r6k7JH0yESkXGZmJlOmTOGcc86hZ8+eHHHEEVW2raycdpT1DB06lM8++4xu3bpx33330alT7CqrgQMHsmvXLrp27cqECRPo06dPjeMfMmQIOTk55ObmMmrUKPr27Rtqs3nzZvr3709BQQEjR47kF7/4BQCPPfYY99xzDz169OCEE07gn//8Z423X1tVlTSHWNfVuHHj6Nq1Kx06dCjv+qrOgAEDWLVqVfkJaYgd8X355Zf7TJcSRCvZPRH4hFgJjW/K5rt7/Y6sEVDJbqlrKtktqVZUVMQNN9xQfpXU3qqvkt1lVybFdyU5oDukRUT20p133skDDzxQ5Sh4qVJtcnD3yi8HEBGRvTZhwgQmTJiQ6jBCEpXsPtXd55rZBZW97+7PVDZfRETSX6IT0qcEz+dV8ohUjMXMBprZu2a2zsyqTI1mNtTM3Mwi94eJ1KXqzr2JpIu6+l1OVLL7tuC5VqfPzSwDuJ/Y4EDFwGIzm+nuqyq0a0rsLuw3a7Mdkb2VmZnJli1bOPzww2t9rbrIvsDd2bJlC5mZmdU3rkaUkt3fAUYB2fHt3f26ahbtDaxz9/XBep4EBgMVyy/+DPglundCUiQrK4vi4uLy+jYi6SwzM5OsrKy9Xk+Uq5VmAQuBd4DSatrGOxLYFDddDBwf38DMegLt3P0FM6syOZjZVcBVAO3bt69BCCLVO+igg6oswyByoIqSHDLd/ca63nBQ6fV/gdHVtXX3KcAUiN3nUNexiIjInqLcIf2YmY01szZm1qLsEWG5zUC7uOmsYF6ZpkB3YJ6ZbQT6ADN1UlpEJPWiHDnsAH4F/ITYzW8Q7Sa4xUCOmXUglhQuAS4te9PdvwBalk2b2TzgZnev3e3PIiJSZ6Ikh5uAY9w9PBZgAu6+y8zGAS8BGcDD7r7SzG4Hitx9Zs3DFRGR+hAlOawDalUG0d1nETuhHT+v0uFF3b1/bbYhIiJ1L0py+ApYZmavsGfhveouZRURkTQVJTk8FzxEROQAEaXw3iP1EYiIiOw7EhXee8rdLzKzd/j2KqVy7t4jqZGJiEjKJDpyGB88RyqyJyIi+49Ehfc+CornTXX3AfUYk4iIpFjCO6TdfTdQambN6ykeERHZB0S5WulL4B0zm03sslZAl7KKiOzPoiSHZ4KHiIgcIKIkhxnA9qCLqWwQn4OTGpWIiKRUlKqsLwON46YbA3OSE46IiOwLoiSHTHf/smwieH1I8kISEZFUi5IcvgpGbAPAzI4Dvk5eSCIikmpRzjlcDzxtZh8CBvwHcHFSoxIRkZSKUltpsZl1AToHs951953JDUtERFKp2m4lMxtG7LzDCuB8YHp8N5OIiOx/opxz+G9332ZmJwLfBX4PPJDcsEREJJWiJIfdwfM5wIPu/gLQKHkhiYhIqkVJDpvN7HfETkLPMrODIy4nIiJpKso/+YuAl4Az3f1zoAXww6RGJSIiKZVosJ9m7r4VyATmBfNaEBtHuqheokszk2b/o87WdcPpnepsXSIiNZXoUtY/EhvoZwmxkeAs7j0HOiYxLhERSaFEg/2cGzx3qL9wRERkX5CoWynhvQzuvrTuwxERkX1Bom6lXwfPmUAhsJxY11IPYucc+iY3NBERSZUqr1Zy9wHB2NEfAT3dvdDdjwOOBTbXV4AiIlL/olzK2tnd3ymbCMpodE1eSCIikmpRksPbZvaQmfUPHg8Cb0dZuZkNNLN3zWydmU2o5P0bzWyVmb1tZi+b2VE1/QAiIlL3oiSHMcBKYHzwWBXMSygYTvR+4CwgFxhuZrkVmr0FFLp7D2LDkf5P9NBFRCRZopTs3g5MCh410RtY5+7rAczsSWAwseRStu5X4tovBEbWcBsiIpIEiS5lfYXYzW6fufuFtVj3kcCmuOli4PgE7a8A/lpFLFcBVwG0b9++FqGIiEhNJDpyGE0sOexO0KZOmNlIYpfLnlLZ++4+BZgCUFhY6MmOR0TkQJcoOcwjlhxKSPyNvyqbgXZx01lUcgmsmZ0G/AQ4xd2/qcV2RESkjiUqn7G3ZTMWAzlm1oFYUrgEuDS+gZkdC/wOGOjun+zl9kREpI4kbVwGd98FjCNW7ns18JS7rzSz281sUNDsV0AT4GkzW2ZmM5MVj4iIRFft1Up7w91nAbMqzLs17vVpydy+iIjUjkZ0ExGRECUHEREJUXIQEZEQJQcREQlRchARkRAlBxERCVFyEBGRECUHEREJUXIQEZEQJQcREQlRchARkRAlBxERCVFyEBGRECUHEREJUXIQEZEQJQcREQlRchARkRAlBxERCVFyEBGRECUHEREJUXIQEZEQJQcREQlRchARkRAlBxERCVFyEBGRECUHEREJUXIQEZGQpCYHMxtoZu+a2Tozm1DJ+web2fTg/TfNLDuZ8YiISDRJSw5mlgHcD5wF5ALDzSy3QrMrgH+5+zHAJOCXyYpHRESiS+aRQ29gnbuvd/cdwJPA4AptBgOPBK9nAN81M0tiTCIiEkEyk8ORwKa46eJgXqVt3H0X8AVweBJjEhGRCBqmOoAozOwq4CqA9u3b13o9N5zeqa5CSsn6RUTqSzKPHDYD7eKms4J5lbYxs4ZAc2BLxRW5+xR3L3T3wlatWiUpXBERKZPM5LAYyDGzDmbWCLgEmFmhzUzge8HrC4G57u5JjElERCJIWreSu+8ys3HAS0AG8LC7rzSz24Eid58J/B54zMzWAZ8RSyAiIpJiST3n4O6zgFkV5t0a93o7MCyZMYiISM3pDmkREQlRchARkRAlBxERCVFyEBGRECUHEREJUXIQEZEQJQcREQlRchARkRAlBxERCVFyEBGRECUHEREJUXIQEZEQJQcREQmxdBs+wcxKgK+AT1Mdy15oSfrGn86xQ3rHn86xQ3rHn86xQyz+Q9098mhpaZccAMysyN0LUx1HbaVz/OkcO6R3/OkcO6R3/OkcO9QufnUriYhIiJKDiIiEpGtymJLqAPZSOsefzrFDesefzrFDesefzrFDLeJPy3MOIiKSXOl65CAiIkmUdsnBzAaa2btmts7MJqQ6nqjMrJ2ZvWJmq8xspZmNT3VMNWVmGWb2lpk9n+pYasrMvmNmM8xsjZmtNrO+qY6pJszshuD3ZoWZPWFmmamOKREze9jMPjGzFXHzWpjZbDNbGzwflsoYq1JF7L8KfnfeNrNnzew7qYwxkcrij3vvJjNzM2tZ3XrSKjmYWQZwP3AWkAsMN7Pc1EYV2S7gJnfPBfoAP0ij2MuMB1anOohauht40d27APmk0ecwsyOB64BCd+8OZACXpDaqak0FBlaYNwF42d1zgJeD6X3RVMKxzwa6u3sP4B/Af9V3UDUwlXD8mFk74AzggygrSavkAPQG1rn7enffATwJDE5xTJG4+0fuvjR4vY3YP6cjUxtVdGaWBZwDPJTqWGrKzJoDJwO/B3D3He7+eWpuq6CXAAAGKUlEQVSjqrGGQGMzawgcAnyY4ngScvfXgM8qzB4MPBK8fgQ4v16Diqiy2N39b+6+K5hcCGTVe2ARVbHvASYBPwIinWhOt+RwJLApbrqYNPoHW8bMsoFjgTdTG0mN/IbYL1ZpqgOphQ5ACfCHoFvsITM7NNVBReXum4G7iH3j+wj4wt3/ltqoaqW1u38UvP4n0DqVweyFy4G/pjqImjCzwcBmd18edZl0Sw5pz8yaAH8Crnf3ramOJwozOxf4xN2XpDqWWmoI9AQecPdjiZVf2Ve7NEKCvvnBxJJcW+BQMxuZ2qj2jscuk0y7SyXN7CfEuoinpTqWqMzsEOD/AbfWZLl0Sw6bgXZx01nBvLRgZgcRSwzT3P2ZVMdTA/2AQWa2kVhX3qlm9nhqQ6qRYqDY3cuO1GYQSxbp4jRgg7uXuPtO4BnghBTHVBsfm1kbgOD5kxTHUyNmNho4Fxjh6XUPwNHEvlgsD/6Gs4ClZvYfiRZKt+SwGMgxsw5m1ojYSbmZKY4pEjMzYn3eq939f1MdT024+3+5e5a7ZxPb53PdPW2+ubr7P4FNZtY5mPVdYFUKQ6qpD4A+ZnZI8Hv0XdLohHqcmcD3gtffA/6cwlhqxMwGEutWHeTu/051PDXh7u+4+xHunh38DRcDPYO/iyqlVXIITgiNA14i9sfxlLuvTG1UkfUDLiP2rXtZ8Dg71UEdQK4FppnZ20AB8PMUxxNZcMQzA1gKvEPs73afvmPXzJ4AFgCdzazYzK4A7gRON7O1xI6G7kxljFWpIvb7gKbA7OBvd3JKg0ygivhrvp70OjoSEZH6kFZHDiIiUj+UHEREJETJQUREQpQcREQkRMlBRERClBwk5czsy0rmXW1mo+o5jnlBxd+3gwqc98VX3zSzN+pgG4Vmdk8Nl3moros0BlVqv1+X65T9iy5llZQzsy/dvUk9b9OI/f6Xxs2bB9zs7kXBTZa/IFYJ9ZQ62mbDuOJtKRXU93o+qPIqEqIjB9knmdlEM7s5eD3PzH5pZovM7B9mdlIwPyOos784+Lb/n8H8Jmb2spktNbN3gqJjmFl2cGTwKLCCPUux7CGo+vsjoL2Z5QfLfxk8tzGz14KboVbExTMw2OZyM3s57nM8ZmbzgcfMrL8F42EE7z1iZq+b2ftmdoGZ/U8Q84tBuZWyz19YFoOZ/f9gGwvNrHUw/zwzezMoLDgnbv5Ei9X3n2dm683suuAj3gkcHXyGX1X1meTApeQg6aKhu/cGrgduC+ZdQaxCaS+gFzDWzDoA24Eh7t4TGAD8OjhSAMgBfuvu3dz9/UQbdPfdwHKgS4W3LgVecvcCYmNDLDOzVsCDwFB3zweGxbXPBU5z9+GVbOZo4FRgEPA48Iq75wFfEyuRXtGhwMJgG68BY4P5fwf6BIUFnySW2Mp0Ac4kVvL+tiDpTADec/cCd/9hZZ8p0b6R/V/DVAcgElFZocIlQHbw+gygh5ldGEw3J/bPvxj4uZmdTKzE+JF8Wx76fXdfWIPtWiXzFgMPB/9kn3P3ZWbWH3jN3TcAuHt8Pf2Z7v51Fev/q7vvNLN3iA3i82Iw/524zxlvB1A2Et8S4PTgdRYw3WIF7RoBG+KWecHdvwG+MbNPqLxUdugzVRGvHCB05CDp4pvgeTfffqkx4Nrg22+Bu3cIxjkYAbQCjgu+CX8MlA2r+VXUDVps5ME8KhS5CwZTOZlYReCpEU6cJ9rmN8E6S4GdcdU+S6n8y1t8m/h9cS9wX3DU8Z98+3nLt1HJMuVq8ZlkP6fkIOnsJeCauL75ThYbxKc5sfEndprZAOComq44WOcvgE3u/naF944CPnb3B4mNjNeT2OhgJwfdWphZi734XLXRnG/L138vUcPANmKF5IAqP5McwNStJPuCQ8ysOG46aknzh4h1vSwNzimUEBt6chrwl6CrpghYU4NYppnZN8DBwBwqH4a2P/BDM9sJfAmMcvcSM7sKeMbMGhAbq+D0SpZNlonA02b2L2Ausfr9VXL3LWY232KD0P+V2An6PT5TkuOVfZwuZRURkRB1K4mISIiSg4iIhCg5iIhIiJKDiIiEKDmIiEiIkoOIiIQoOYiISIiSg4iIhPwfMqCifs6qydoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff3a52b8a90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.bar(range(1, 14), discr, alpha=.5, align='center', label='individual \"discriminability\"')\n",
    "plt.step(range(1, 14), cum_discr, where='mid', label='cumulative \"discriminability\"')\n",
    "plt.ylabel('\"discriminability\" ratio')\n",
    "plt.xlabel('Linear Discriminants')\n",
    "plt.ylim([-0.1, 1.1])\n",
    "plt.legend(loc='best')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以看到前两个线性判别捕捉到了Wine训练集中100%的有用信息。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然后由这两个线性判别式来创建转换矩阵$W$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "w = np.hstack((eigen_pairs[0][1][:, np.newaxis].real,\n",
    "              eigen_pairs[1][1][:, np.newaxis].real))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('Matrix w:\\n', array([[-0.0707,  0.3778],\n",
      "       [ 0.0359,  0.2223],\n",
      "       [-0.0263,  0.3813],\n",
      "       [ 0.1875, -0.2955],\n",
      "       [-0.0033, -0.0143],\n",
      "       [ 0.2328, -0.0151],\n",
      "       [-0.7719, -0.2149],\n",
      "       [-0.0803, -0.0726],\n",
      "       [ 0.0896, -0.1767],\n",
      "       [ 0.1815,  0.2909],\n",
      "       [-0.0631, -0.2376],\n",
      "       [-0.3794, -0.0867],\n",
      "       [-0.3355,  0.586 ]]))\n"
     ]
    }
   ],
   "source": [
    "print('Matrix w:\\n', w)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 原始数据映射到新特征空间\n",
    "\n",
    "有了转换矩阵$W$,我们就可以将原始数据映射到新的特征空间了：\n",
    "\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576d24cc6333f.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_lda = X_train_std.dot(w)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "colors = ['r', 'b', 'g']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "markers = ['s', 'x', 'o']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3XtsHVedB/Dvz07SJNSxg9OKYtd2pa1oQktNk21AIFalPArb0qURiBJBSyq8SKW5SZBYVl7ViVDYXaomcQsVyi4FVFkFpMLyatMnFbsVBRzqkpbwyELSOqCSenGSkoQkvr/9Y+7Yc8czc+d9Zu79fqSrZMb3zhzfpuc35/zOQ1QVREREbaYLQERExcCAQEREABgQiIiohgGBiIgAMCAQEVENAwIREQFgQCAiohoGBCIiAsCAQERENQtMFyCKFStW6MDAgOliEBGVyt69e19W1fMava9UAWFgYADj4+Omi0FEVCoicijM+9hlREREABgQiIiohgGBiIgAlCyHQERkwpkzZzA5OYlTp06ZLkqgxYsXo7e3FwsXLoz1eQYEIqIGJicn0dHRgYGBAYiI6eJ4UlVMTU1hcnISF110UaxrsMuIiKiBU6dOobu7u7DBAABEBN3d3YlaMQwIWVq2DBCZ/1q2zHTJiCiiIgcDW9IyMiBk6fjxaOeJiAxiQCCiljG2bwwDuwbQtq0NA7sGMLZvzHSRItmwYQPOP/98XHrppZlcnwGBiFrC2L4xDH1vCIeOHoJCcejoIQx9b6hUQeHmm2/Gnj17Mrs+AwIRtYThx4dx4syJunMnzpzA8OPD6d4ow9zh2972Nrz61a9OoZDeGBCIqCW8cPSFSOdjK3HukAEhSx0d0c4TUWb6OvsinW9FxgKCiFwoIj8UkV+KyPMiUjFVlswcOwaozn8dO2a6ZEQtZ/vV27F04dK6c0sXLsX2q7cbKlHxmGwhnAXwKVVdBeBNAG4VkVWp34VzAYgIwPrL1mP3dbvR39kPgaC/sx+7r9uN9ZetN120wjC2dIWq/hHAH2t/Py4i+wH0APhlqjcqcX8eEaVr/WXrIweAsX1jOP/E+fjLH/6CRe2L0NPRg+6l3RmVMNiNN96IJ598Ei+//DJ6e3uxbds23HLLLaldvxBrGYnIAIA3AviJ2ZIQUasa2zeG4ceH8cLRF9DX2TfblTT0vSE8cNUDAIDTM6dx6Ki114xvUOjo8H7gTCF3eP/99ye+RhDjAUFEzgXwAIBNqjqvc11EhgAMAUBfH5M/RJQ+e46CPSzVnqOwZMGSeUNVq1rF4eOH/QNCiXOERgOCiCyEFQzGVPVbXu9R1d0AdgPAmjVrNMfiEVGL8Juj4D5nOz1zGlMnpnD4+GGcnjmduCspzWslYSwgiLUK05cB7FfVHabKQUQUdS7CgrYFOHT0EKpaBRCyK8nH1Imp1K6VlMlRRm8B8BEAbxeRidrrvanfpdnmAnDUFFHq/OYidC/pnjdUtU2satOuwG12V1JUh48fTu1aSZkcZfQ/ALJfTzbv/rxly/wTSmmUhaOmiFK3/ertdTkEwJqjMPqeUQBA+4l2AJjtzvn99O89r3N65nTke/t9Js61kjKeVG46rLCJSsceiuoeZWSf379/P1a+duXs++3+frdF7Ysi33tR+6LUrpUUAwIREaLNUejp6Knr9wesrqSejp7I9w17rRdffBEf/ehH8dJLL0FEMDQ0hEol3QUeGBCaiUh6XVNE5MtO9vqNDFK1/ne0uY+jXMu2YMEC3Hnnnbjiiitw/PhxrF69Gu985zuxalV6CzwwIDQbdk0R5aJ7abfnKKCtW4HpaWDnTisIqAKbNwNdXdbPolzL6YILLsAFF1wAAOjo6MDKlStx+PDhVAMCVzstm7KOjiJqAapWMBgdtYKAHQxGR63zmtJMqoMHD+KZZ57B2rVr07lgDVsIactw2jqAue6gEmz4TdTM/CaT7dxp/Xx01HoBQKUy12JI6pVXXsG6deuwa9cuLEt5uDlbCGnjktdETc+eTGaPDrInk02dmIIIZoOCLa1gcObMGaxbtw7r16/HDTfckPyCLgwIREQRBU0ms7uJnOzuoyRUFbfccgtWrlyJLVu2JLuYDwaEsmq2GdhEORjbN4aBXQNo29aGgV0DGNs3Fus6fpPG/nr29GzOoFIBqlXrT2dOIa6nnnoK9913H5544gkMDg5icHAQDz74YPwLemAOoazYBUUUid+KpgAi75HgN5lMBDjR/ges//hC/Mv2NojM5RS6upJ1G731rW+FppWV9iFZ3yBNa9as0fHxcdPFIKISGtg1MLtonFN/Zz8Objo477xzf4SHr3kYV1x2xezQUPeCdG6qQHtbG/o7+9G9tDtwHkLa9u/fj5UrV9adE5G9qrqm0WfZZVQUfovWcfE6olT4rWjqdd5uTRw6eggKxUx1ZjZpDFjzBvo7+32XlxCpX6CuLIMCGRDCyGOF0UYTyjjhjCgRvxVNvc5XHqr4bowzdWIKv3jpF7ML3F3UdZHvPU0sUJcEA0IYXLCOqPS2X7193lLWSxcund0q0za2bwxTJ6c8r2EPL3UPN13Q5p2ONbFAXRIMCGXCLiSi2NZfth67r9uN/s5+CAT9nf3Yfd3ueQnl4ceHA6/jNdzUL5fQeU5nskLnjKOMyoqtE6LIwqxoGnX3NGB+kLAd/evRyNcyiS2EtJjcyaxRQpotC6LQ/HINC9oWRO4CSjOHcOrUKVx55ZW4/PLL8frXvx4jIyOpXdvGgJAWk3mGKPdgy4IokFeuQURw4bIL0dPRM7uF5uzPAjZ+TDOHcM455+CJJ57As88+i4mJCezZswdPP/10atcH2GXUWNATdZqzgv0WxSOiXHntnta9pH55aueidlWt4mz1bN01HjrwEO752T146ZWX5u2+1ojfonkignPPPReAtabRmTNnICmPZ2VAaCSokk5ztrDftcoygJmoibhzDfv375/9u3vvgvE/1E+WfejAQ/jcf38Op86eAhBtRrR7wps9ism+78zMDFavXo0DBw7g1ltvTX35a3YZFR3XLCJK3di+Maz4/ArINoFsE6z4/IrY6xq5u4Xu+dk9s8HAduLMiYajl4DgRfMAoL29HRMTE5icnMRPf/pTPPfcc7HK7Ke1A0KeieC49+Jy2kSpGts3hg3f2VA312Dq5BQ+9l8fixUU3HmFl155yfN9YUYv+SWh3ee7urpw1VVXYc+ePRFK2lhrB4Q0E8GNnuSzTDpHaS2wZUEtbvjxYc+K90z1TKineDf3MhavOfc1nu/zG73k5JeEXtS+CEeOHMH09DQA4OTJk3j00UdxySWXRC5vEOYQ0pL2E/uyZf47r7nvxdYCUWhBT+px5iAA9XmFO951R92qqoD3jGgvPR098xbNa5M29HT04PCBw7jpppswMzODarWKD37wg7j22mtjldcPA0IjWW+J6YfLZRBloq+zz3PVU/tnSXmNUgo7ysgOKl6jjLrf0I1nnnkmcfmCGA0IInIvgGsB/ElVLzVZFl98+ibKhXO56ahDNaPYfvV2bPjOhnndRgvbFoZ6ig8jzIxoP+5RTHkynUP4KoBrDJeBiAxzLzdtD9WMO/InyPrL1uPe6+9F95K5Srd7STe+8g9fySQAlYnRFoKq/khEBowVIM/uIFNdT3FEyV8QpWD48eF5y03bQzWzqKTjPMGraqyJYH4TzbKQdMMz0y0Es/Ic0lmm4aPMX1DOomxek1ScfZUXL16MqampyBWuPdHMvVy2vdFOmlQVU1NTWLx4cexrFD6pLCJDAIYAoK8vecKnNJK0KOI84ft9higHfoneNJK8TnH3Ve7t7cXk5CR+f/j3mD41jZnqDNrb2rF88XK8atGrfD83eWwSM9WZeeen2qbQu6w34W8z3+LFi9HbG/+6xvdUrnUZfT9MUrkweyoXvUslqFnr9987bFO4RHtwU3m4K2rAGqrptV9BElH3VU5axrZtbVDM/39GIKiOeC+ZnQXuqZwldqkQpSrs5jVJJemaCspz+ImybWcRGA0IInI/gB8DeJ2ITIrILSbLQyFwXwWKqVHf/frL1uPgpoOojlRxcNPBTJLJUStoZ5n95i4EBZOw23YWhdGAoKo3quoFqrpQVXtV9csmy1NIJjfeCcLWEEUQdVipV/CIkwx2i1JBu8vsJ+hpP6+WT1qM5xCiKEwOIU4ffRb3crNzGFnmENz3K3IuhQojSt+9V1/9ovZFUFWcqZ6ZPScQfGLNJ3DP398TqSxhJ8D5ldkpizxHFsLmEBgQ4ihqQLDvX4RRRiX6d0XZi5JcDVMROz9/3w33ZVIh+5XZvm+Ws6nTxqRyloq+R0GcOQ9+nyFKQZS++yhzDxQaa4XSMPzK3N/Zn2mewyQGhDjKNMmMqACi9N1HHYGTxeQ1oHwJ4TQwIGSlqMngqIreGqJSiJJc9aqIgzarz2oIZ9kSwmlgDiEraeUZovbth7l2lBxD0HuDylWif1dUPF6J36deeApfGv9SXb9+WZK6pjGpbFqWieekM6WjlC3ovRxlRDnLa4nsZsOAYFqeI5GiSisgmP49iCgUjjJqFc2SqyDKSBoT2lpF4Vc7pQa4rhKRr7irm7YqthCywtE5RKmK86QfZ0G6vMpWRGwhZKXISdUoey2Uaac3alpxn/Tz2HinmVohbCGUiVe+II4oE+s4CY8KIO6Tfh7LT+fVCskDA0IjRUralj0vUKTvkkol7pN+HrONs2iFmOqCYkBopKxJ2yJ26ZT1uyTj4j7p27ONu5fMbWq/ZMGSQpTNT9SlwtPEgNCMODGMmkzSJ/2TZ0/O/n3q5FSqFWzarRCTXVAMCM3I+cTNbppSc8/9K8tcwLS7POKuKzS2bww3ffsmzwq28lAllTKmveZRHolwP5yp3EiRZupGSSLbZStL+YPKknSpjpLauhWYngZ27rS+OlVg82agq8v6WVHF2Yw+r3IEKcq6SFE2EwqLM5WbURHzAnlowdyDqhUMRketIGAHg9FR63yRn+OKMurGqxxBijIyyOSy25yH0EiRxuG7n4bjDjs1pUjfZcGJWC0DwAoCo6PW3yuVuRZDUZns8kh6v7zL6MVuoZhYxI9dRmUWpgumSF1GcTXD7xCTKtDmaMdXq8UOBkA2XR5plqNd2tG1uAtTJ6fm/SzvMuaFXUatgMtjNDW7m8jJ7j4qsqLsNOZXjq+9/2sYfc9oIcpYNAwIZdZoFnHQSKI8ggZHOMXmzBlUKlbLoFKpzykUVVF2GgsqR9gyNssaRWGxy6iZme5qyXrXOI4yopDibKxTlNFSaeAGOc0qSuXYLAGhhanWf43uY2osbsVelFxIGkqRQxCRa0Tk1yJyQEQ+Y7IspZHWEEx245SCu/JnMIgu7jDYooyWypOxgCAi7QC+COA9AFYBuFFEVpkqT8tr4jH91NriVux5rJRaNCZbCFcCOKCqv1PV0wC+DuB6g+UhoiYUt2IvymipPJkMCD0AXnQcT9bO1RGRIREZF5HxI0eO5Fa4phB1JFHa3UYcFksFELdiL8poqTwVfqayqu4GsBuwksqGi1MuXiNwgjqh0+42auIRQFQeSWb+2kNUW4XJgHAYwIWO497aOQrC5R+IImu1ij0uk11GPwNwsYhcJCKLAHwIwHcNlqcckm5pycBBRD6MtRBU9ayIfBLAwwDaAdyrqs+bKk/LOHaMYxeJyJPRHIKqPgjgQZNlICIiC9cyakUc/UMuZd2ZjdLFgNCKkuYhqKls3Vq/YJ69ZhLXS2o9DAhETSjsE3+Zd2aj9DEgEDWZKE/8IkBnJzA4aAWBtjbrz8FB6zzHH7QWBgSiJhL1iV8VOHoUmJioPz8xYZ2P0kJgHqL8uPw1UZNxBgFb0F7M1SqwenV9UBgcBPburd++Mwj3bii2Uix/TUTpE7EqZie/YKAKbNni3ULYsiXcUz7zEM2DAYGoyUTZi9mZQ3CKkkOwA5C9xaedhwhqlVAx+QYEEVkmIv8qIveJyIddP7sn+6IRUVRR92J25hCc74+aQ4jSKqHiCpqp/BUAvwXwAIANIrIOwIdV9a8A3pRH4YgoGhGr3975dG5X1F1d3juwRXm/H79WCYNCufgmlUVkQlUHHcfDAN4L4H0AHlXVK/Ip4hwmlYnCiboXc5K9m92tkp075x8zKJgVNqkc1EI4R0TaVLUKAKq6XUQOA/gRgHNTKicRZSDqXsxJ9m5Oq5VB5gW1ED4P4BFVfcx1/hoAd6vqxTmUrw5bCETFlaSVQdlK3EJQ1U/7nN8DIPdgQGQKK7pwkrQyqBg47JQoABd+o1bCgEDkgxOuqNUY3SCHqMicydHR0bmlIDhyhppV4FpGItIN4MMALqmd2g/gflWdyqFs8zCpTCao1q/pU60yGFC5JF7LSERWAngOwGoAv4E1Se1vAewTkUv8PkdUNElW4YyyDARR2QXlED4LoKKqN6vqqKruUtWbANwGYHs+xSNKJklSOOoyEERlFxQQLlPVb7pPquoDAC7NrkhE6UiaFPabcFWpcMIVNaegiWk/91ueIuhnWWIOgaKKujeA3zWymofAOQ6Uh7A5hKCAMAlgh9ePAGxS1QuTFTE6BgSKo6hJYW4qQ3lJY4Oc/wDQ4fE6F8B/plFIah2mtlcsalI4SncWt6akvAQtXbHN72cisimb4lAzMvUkHLQKJ2B2LkHYOQ5sRVCe4s5U3pLkpiLyARF5XkSqItKwGUPlZXK2b9pJ4bSf1BttKsOZ0pQ7VY38AvBinM85Pr8SwOsAPAlgTdjPrV69Wql8qlXVSkXVqsKsV6Vinc/r/kHHYYyM1JfZ/p1GRpKVq9H3Yvq7o+YAYFxD1LFxWwiJnk1Udb+q/jrJNag8TG+vmHQVziye1MPOcTD93VFr8c0hiMhxeFf8AmBJZiWaX44hAEMA0NfXl9dtKUVl314xTH9/1OGjYTeVKft3RyUTphkR5wXgMVhLX7hf1zve8yTYZdTUnF0edleH+7iIvLqZqtX6rhv7PXG7k+xrBh2X8buj4kHILqPMVjtV1XdkdW0qjzjbK0Z92k6b18ieTZuAn/yk/n2bNwM7dsx1JwHz9xP2K7vzHoD36CFuTUm5CxM1snqBLYSWETaxm0XyNgqvp/CNG+daBRs3zn/PzEy0xG/UJ/80kuLU2mC6hRBERN4P4G4A5wH4gYhMqOq7TZSF8hEmsetM3gLhn7bTLqdXvmDtWuu1a9f8J/W2NuvYuTxGUB+/CNDZCQwO1t9jcNA63+i7YsuAMhMmahTlxRZC8yvKMEuvfIHfk3rUMnu9n0NKKUvIeNgpUSb8hlm6acyBz+7PeV1HfUb2uDlnDkdZIlvEyj0MDtafHxy0zrMFQKYwIFCheFXGb36zldS1K1f7PVGXbgizN0LUCj7ObGhVYMsWYGKi/vzEhHU+brAzKUygpeJjQKBcBVUcXpXxxo3W6J677poLCnEmhVWr9ZPLqlXv64Sp4N33HBmpzxnYn/ELWM4cgpNfDqHokmxCRAUTpl+pKC/mEMotzAgir/ds3Ki6dm38vnb7mlFGA/nlC9IYBdVM8wua6XdpZgiZQzBeyUd5MSCUV5SKI8qksKj3nZmpv87MTDa/QyOmh9emqSgDAcgfAwIVTtyKI2mFk+aonjQrv2aaXxA3YFM+GBCokKJWHGk9lXu1DOI+3bPyq8cWQvGFDQhMKlNu1Gc4pwYkhtPY00Bro3qctmyxhngGJYu9yhXndzAhzO+S1n2iDrulAgsTNYryYguhvJI+6cftXglaZsI+Vg3Xp1+WBGre+Ylmyoc0KxR56QpqPUkXaou7fEOj+7a1hV8yI+ha7uGi9mfyFvZ3SdPWrfXXtb+Xsg2fJUC0RG26NWvW6Pj4uOliUALuCimvitN5H/ufvPvYud8yMH9/Y69rAdY8hKNH4+97nPZ34uzGafS7UGsQkb2q2nC7YuYQKFemFmpzblrvzAHYlee2bfOXyHAvI+EOJPa5o0fj76aWxaQu7rJGcTEgUEuwx7/Y3SnuWc9//rN1zmn1aitJan/eq6J2JrlHR60uKLt7JqgSblSeuFtzOsvqxAQvhRIm0VCUF5PKFIdzprJ7fwPAOrbP2e8bHLSOBwfDDVGNMhTVmYT1Kk+SBHVZEt+UL3DYKVH9U/jq1d5Pybt2AcuXzz3Vt7UBe/daawtNTADt7cFP/X5P5Hbrwvk+Z3m8VlAFknXvpDFMl1pYmKhRlBdbCBSH84nf62Xvgube88A9mc2vZeD3RG63LpzvGxkJnjmd1pN8M82CpuTAFgKRxX7id9u40XrddddcfsDua/d76ne3MESsVoRzL4M77wRWrKhfztqZGwDmJ303bgw/qct93ut93GWN4uA8BCo1DTFkUz1mKgNzXSr2ctT2aCG7gr3rLuvPjRutP51j+51DVu2tMLdssX72qU8BL788f4tMuxsHCO4uAvy7d9xDXKtV675hh7gSBQrTjCjKi11G5BR1drG728ju0gnaCtPZneQ3+9ZvLR+vLievLiZnQttrq07b7bfP/Q7u5DcTxhQEXNyOmlmU0TRRKtKg0UJBFa77c0F7L8RZ6iFMYCPyw4BATS/KKpu3317/M7vC9mtNREnwen3OHXDcwSpO0tcvGc1gQI0wIFBLiPJE76443e+NM34/ziijJNzdUOwuojDCBgQmlam01Gck0M6d1lIU09P16wu5k6/OpG3cxffCLngn4r0URpTRP9WqNZfCyU5cA1yeglIQJmoU5cUWAtmCnszdM4+jzNZNssy2+9iZKxgZmSuXcy5C2BaDV8vDeXz77eGuQ60JRW4hiMgdAK4DcBrA/wL4mKpOmygLlVOjJ/qREeuc17DPoKfoJMtsu9kzktVjGOumTdZx2CWpnb/vjh3W3ApnS2TbtnDlJApiZPlrEXkXgCdU9ayI/DsAqOo/Nfocl78mN3dl6jxWtSpO28xM/XHULps4ZXMvQ+0UZ0nqoN+XyE+hl79W1UdU9Wzt8GkAvSbKQeXn90TvlV8Is3pp2mVzz0h2itPnzxnIlKUiLF2xAcBDpgtBzcP5ZF6pWC0De6E6Oyikscx02HL44ZLUVDSZ5RBE5DEAr/H40bCqfqf2nmEAZwGMBVxnCMAQAPT19WVQUmo2XvmFvXutYGCvXgpkt4uYXcnbQcde+iLMUhhEJhnbQlNEbgbwjwCuVtUTYT7DHAJF4e5fr1bngoF9nMX+wvZw123brI13AODZZ4HLL7f+vny5lfSOss0mURJhcwimRhldA+DTAP4ubDAgisqdfHUvcGfPWUgrKKjO3+DeOZrInodg368oLQMmqslmKofwBQAdAB4VkQkR+ZKhclALcOcUwi4zHfUeXttp2sHA3njHPRnOtCz2dKbyMtJCUNW/MXFfak1xZyGH5ewmsmckO4eaBo00MsmrReMMnGwptB4uXUEtYevW+grODgpJKzx3pbpjB9DTU/+eTZusP5cvL9aTtzMwRp3AR83JWFI5DiaVqYj8JqDddptVqTpHF+3aVbyK1j2BL4tkO5lV6IlpRM3EawLabbcBd989FwzWrs02GLif68I+5/ktEFii50RKEQMCUUJelaq74v/xj7239gw6DituYjiPZDuVCwMClU5aFWkavCrVjRvnWgY2dwWb1ugeZw7Dvl7YWdh+yfZKJZ1kO5UPk8pUKu4RPXYFaGqCl7tSdVq71moZOPML9nvSGt2TNDGcVbKdyokBgUqjqMMk3ZXq8uX1CWSvIa5pju6x7+Ee6hp36W4Gg9bFUUZUKl4jeoo4TDLM7N+0RveU5TshczjKiJqS14ieIlZ8jZ660xrdw8QwpYkBgUqlGYZJplmJMzFMaWIOgUrDXZE6cwhAMVsKXtJeSoOJYUoLAwKVRtZrEuUp7UqciWFKA5PKVDpcrpkoGiaVqWnxaZgoGwwIREQEgAGBiIhqGBCIiAgAAwIREdUwIBAREQAGBCIiqmFAICIiAAwIRERUw4BAREQAGBCIiKiGAYGIiAAYCggi8lkR+YWITIjIIyLyWhPlICKiOaZaCHeo6htUdRDA9wHcbqgcRERUYyQgqOoxx+GrAJRnDW4ioiZlbIMcEdkO4KMAjgK4ylQ5iIjIklkLQUQeE5HnPF7XA4CqDqvqhQDGAHwy4DpDIjIuIuNHjhzJqrhERC3P+I5pItIH4EFVvbTRe7ljGhFRdIXeMU1ELnYcXg/gVybKQUREc0zlEP5NRF4HoArgEIBPGCoHERHVGAkIqrrOxH2JiMgfZyoTEREABgQiIqphQCAiIgAMCEREVMOAQEREABgQiIiohgGBiIgAMCAQEVENAwIREQFgQCAiohoGBKKI3AsEG14wmCg1DAhEEWzdCmzePBcEVK3jrVtNloooHQwIRCGpAtPTwOjoXFDYvNk6np5mS4HKz9gWmkRlIwLs3Gn9fXTUegFApWKdFzFXNqI0GN8xLQrumEZFoAq0OdrW1SqDARVboXdMIyoru5vIyZlTICozBgSikJw5g0rFahlUKvU5BaIyYw6BKCQRoKurPmdg5xS6uthtROXHHAJRRKr1lb/7mKhomEMgyoi78mcwoGbBgEBERAAYEIiIqIYBgYiIADAgEBFRDQMCEREBKNmwUxE5AuBQTrdbAeDlnO5VNvxuvPF78cfvxl8e302/qp7X6E2lCgh5EpHxMON2WxG/G2/8Xvzxu/FXpO+GXUZERASAAYGIiGoYEPztNl2AAuN3443fiz9+N/4K890wh0BERADYQiAiohoGhAAicoeI/EpEfiEi3xaRLtNlKgoR+YCIPC8iVREpxAgJk0TkGhH5tYgcEJHPmC5PUYjIvSLyJxF5znRZikRELhSRH4rIL2v/H1VMlwlgQGjkUQCXquobAPwGwD8bLk+RPAfgBgA/Ml0Q00SkHcAXAbwHwCoAN4rIKrOlKoyvArjGdCEK6CyAT6nqKgBvAnBrEf7NMCAEUNVHVPVs7fBpAL0my1MkqrpfVX9tuhwFcSWAA6r6O1U9DeDrAK43XKZCUNUfAfg/0+UoGlX9o6r+vPb34wD2A+gxWyoGhCg2AHjIdCGokHoAvOg4nkQB/uemchCRAQBvBPATsyXhFpoQkccAvMbjR8Oq+p3ae4ZhNfHG8iybaWG+GyLn2pBbAAABzUlEQVSKT0TOBfAAgE2qesx0eVo+IKjqO4J+LiI3A7gWwNXaYmN0G303NOswgAsdx721c0S+RGQhrGAwpqrfMl0egF1GgUTkGgCfBvA+VT1hujxUWD8DcLGIXCQiiwB8CMB3DZeJCkxEBMCXAexX1R2my2NjQAj2BQAdAB4VkQkR+ZLpAhWFiLxfRCYBvBnAD0TkYdNlMqU28OCTAB6GlRz8pqo+b7ZUxSAi9wP4MYDXicikiNxiukwF8RYAHwHw9lrdMiEi7zVdKM5UJiIiAGwhEBFRDQMCEREBYEAgIqIaBgQiIgLAgEBERDUMCEQBROQVj3NbReRwbajgb0XkW34Lk3FVWCoTBgSieHaq6qCqXgzgGwCeEJHzPN7HVWGpNBgQiBJS1W8AeATAhz1+xlVhqTQYEIjS8XMAl5guBFESDAhE6RDTBSBKigGBKB1vhLWOEVFpMSAQJSQi6wC8C8D9pstClAQXtyMKICJVAH9wnNoBYBmAjwM4AuBVsEYSDavqLz0+/34AdwM4D8A0gAlVfXfW5SaKgwGBiIgAsMuIiIhqGBCIiAgAAwIREdUwIBAREQAGBCIiqmFAICIiAAwIRERUw4BAREQAgP8HOHnC4OBq4R0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff3a52aebd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for l, c, m in zip(np.unique(y_train), colors, markers):\n",
    "    plt.scatter(X_train_lda[y_train==l, 0],\n",
    "               X_train_lda[y_train==l, 1],\n",
    "               c=c, label=l, marker=m)\n",
    "plt.xlabel('LD 1')\n",
    "plt.ylabel('LD 2')\n",
    "plt.legend(loc='upper right')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "新数据集，明显线性可分。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 调用sklearn中LDA\n",
    "\n",
    "我们通过一步步地实现LDA来加深理解，现在看看sklearn中如何使用现成的LDA："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.discriminant_analysis import LinearDiscriminantAnalysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "lda = LinearDiscriminantAnalysis(n_components=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_lda = lda.fit_transform(X_train_std, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "逻辑斯底回归 建模"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr = LogisticRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr = lr.fit(X_train_lda, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3X+UXXV57/H3c2YyM4RJiCGJsSQkQAyECNaSBC0aL06gSlFq8bbYq/2B90a5rQuLa2lDVteqqxa8114qqz9YzRV7l5WCtGq1rVLzAx2vdxESohYhGAJoSEoWBJqQSTI/z3P/2GfP2efMmfP7nL33OZ8XK4uZM+fHNyHsZ3+f5/l+v+buiIiIZOIegIiIJIMCgoiIAAoIIiKSo4AgIiKAAoKIiOQoIIiICKCAICIiOQoIIiICKCCIiEhOb9wDqMWiwUFfee65cQ9DpPmOHeOFZYtYMDgY90ikAz2578lj7r640vNSFRBWnnsue7dujXsYIk336VVZ1l25kvN6z4t7KNKBLh+4/GfVPE8pI5GEUDCQuCkgiIgIoIAgIiI5qaohlDLR28vhCy9kdO7cuIcyq4HTp1n27LPMmZyMeygiIrNKfUA4fOGFzFu+nJXz5mFmcQ9nBnfn5ZMnOQxccOBA3MMREZlV6lNGo3Pncm5CgwGAmXHuvHmJnsFIvD5w+/K4hyACdEBAABIbDEJJH5/E64LPvsC6K1fGPQyRzggIIiLSOAWEJrj51ltZcumlvGHjxriHIiJSt9gDgpn1mNkPzOyf4x5LvX77ppt46IEH4h6GpFT/inEtSpNESEKX0a3AfmB+yz/p2mvh2LGZjy9aBN/+dt1vu/Etb+Gnhw41MDARkfjFGhDMbBnwy8CfALe1/AOPHYNSm+OVChIiIl0m7pTR54BPANmYxyESC7WcSpLEFhDM7HrgRXd/rMLzNpvZXjPb+9LISJtGJ9Iei3/9OKvWLYh7GCJAvCmjq4D3mNl1wAAw38y+5O4fiD7J3bcB2wDWrVjh7R9mC91xB5w8OfPxefPg9tvbPx4R6WqxzRDcfYu7L3P3lcBNwK7iYJAW7//wh3nLddfxk4MHWfbGN3LvffdV98KTJ2FwcOavUkFCRKTFktBl1D6LFs3eZdSA+//6rxt6vXSvc288wdr+q+IehgiQkIDg7t8BvtPyD2qgtVREpNPF3WUkIiIJkYgZgkg3+sDty7nkwaqOupUqHJs8xqHJQ4z6KAM2wPm957Oot7F0cLdRQIjTvHmzdxlJV1DLaXMcmzzGgYkDZMjQSy/jPs6BieD8EQWF6ikgvPACZEusi8tk4HWva+1nq7VUpCkOTR4iQ4Ye6wGghx7w4HEFhOopIGSzwcW/1OMiLbT4148DOiujGUZ9lN6iy1mGDKM+GtOI0klF5SZ4/sgRrn7ve7n0rW9l7dvext3btsU9JEmBoOV0bdzD6AgDNkC2aAecLFkGbCCmEaVT180Q3CF6gJl74/dovb29/K9PfYpfuPxyTo6McMWmTVzz9rdz6cUXN/jOIlKN83vPD2oGHswMsrl/zu89P+6hpUpXBYRtXxzg5Ihx2y1nMAuCwV1/u5h5g87m971S9/u+7rWv5XWvfS0A8wYHWbN6NUdeeEEBQaRNwjpBPV1G6k7K65qA4A4nR4z7v9YPwG23nOGue87i/ofO5v3vOj5j5lCvnx46xA8ef5wrr7ii8TeTjqWW0+Zb1Luo5gu5upMKdU1AMAuCAMD9X+ufDgzvf9dL3PaBFzEHolvnlSo0VzAyMsKNN9/M5/74j5mv1lGpYP5ytZy2W/FsYDw7TsbUnRTqmoAA+aAQBgOA2z45B7PGjy+cmJjgxptv5r/ceCO/ev31Db+fiDRXqdnAaU7T7/3TAQG6uzupqwKCO9x1z1kFj911z1nTNYX639f50Mc+xprVq7ntllsaHGWNtIV2KgXpIs0Q2qnUWoWMZ5hggj76pp8X7U5qRn0hTTWKrgkIYTC4/2v9vP+9Y/kaQqSmUG9Q+P7u3fzt3/89l61Zw89ffTUAd2zdynWbNuUXvh0/Dlu25F/UrAt2uIV2qccl0a4+Xy2n7VRqrUIffYwyypRPzehOakZ9IW01iq4JCGYwb9Cng0G0pjBv0BuaIbz1zW/GX3yx9A/DhW9mhRfuVlywjx6Fqanga/d8ANJsQSSoGfh4UCfIyViGszmbOTZnxh38vtF9Da9+TtsK6q4JCACbf3O0oJsoDArN6C5KhKkp6OnJfx0GIM0WRGZdq7B6zuqSF+dmrH5O2wrqrlupXHzx75hgIKmx7c6X4x5CV1rUu4jVc1bTZ31MMkmf9c0aDKA5q5/TtoK6q2YIIomxQAXlONSyVqEZq5/TtoJaASHtoltou+drCD09s79GRCoKA8czE89wyk9hGAPUdmffyArqOCggtFomExSW3WFkJP94sxauRYvFW7aU7jiSRHnx3tfEPQSpwRRTnGVnTd/h19olVM8K6rgoILRaeKbC+DjceWdrP0sH7qTGDWvVcpoGaesSapQCQhOMjo6y8YYbGBsbY3Jqivddfz2f+uQn2z8QtZaKNFXauoQapYDQBP39/ez6ylcYHBxkYmKCt7773bxraIg3r1sX99BEpAGl1i4kuUuoUV0XEH5q/85jtp9XbYT5PsgVvoaV/nMNvaeZMZjL3U9MTDAxMYGpn1VKufdeuPN34h5Fx2r2NhHRLqGsZxlnPGgj9eCzOi1t1FUB4af27zyc2UMPPfTTxyk7w8O2h6uz6xsOClNTU1yxaRMHn3uO37355uZsf33HHXD4cL5zKKqnB5YtU5oojQY68+4ybq3YJiLaaTTKKBky9NMPRqK3oKhXVwWEx2w/PfQwJ/fbDv/9mO2vLyCE+xQBPcAPv/Qljp88yXtvu40f79/PG9asaWzA0QJx8TFvxT8X6XK1FIDDmcSp7CkcxzDOzpw9PaMonmm4O3NtbsGuqJ1YXO6qlcqv2gi9FPbn99LDqzYyyysqCPcpivxacM45XL1uHQ/t2tWEEUun+bTSRS0z6sEdfFSpAnA4kzidPc0kk0wxxQQTnMme4cDEAZ4de5YDEwcY9/GCbbKnfKrie6ddV80Q5vsgp+zM9MwAYJIp5ntjvfsvvfIKc3p7WTB/PmdGR9m+ezef/PjHC5904kThbqehWjae88gJPlNT+R1UtXldqtyg0/RaotoCcDiTmGACAMv9M8kk/fRzeOowfdZXMNMwN8YYCw7UIcMc5pCxTMcVl7sqIFzha3jY9gDBzCC4N5jiCq8jtfPCCzA5mfvyBX7rj/6IqWyWbDbLr117Lddfe23h892bv011uINqpffQmQnSBardJiJsJS21x1CGDJNMFqxInvRJPHKcYpYsY4zR672smrOqtb+pNuuqgLDSf46rs+ub02WUqx1gxuWrV/ODv/u74Ht36E3YH6vOTJAuUO02EeFMIgwaofD7MFiEM41xH5+eRRhGliyG0Wd9HVU/gC4LCBAEhUY7ilqi1F38iROF5xsUi+5XVG4WIMkwPAwXvz3uUXS0araJCGcSPfSQJYvn/umnnyxZlvUs42j26PRMY4rg/0HHcx2K/fRaL5NMNjzepJ2mFltAMLPlwBeB1xIcb7/N3e+Oazx1MSt9oc7UUasvdRc/OBjsfxRueVFuryLNAkSqEp1JRLuMzsqcNX1Bnj85f/rnoXB2MMYYWc8yNzO3oXEk8TS1OGcIk8DH3X2fmc0DHjOz7e7+ZK1v5O7NWwgWaSUtkMnk9yUKzZkz83nZ7IznuXvp9xSRWFSaSYQ/3ze6jzPZM4wTpI0gmClMMtnwFtZJ3CcptoDg7i8AL+S+Pmlm+4HzgJoCwsDp07x88iTnzpvXnKAQtpKWerwO7s7LJ08y8PzzhbudhmpJ6YyMBJ1FxXp7tctpCnz65reWvomQxBr1UebYHDJkGPfx6fpBDz0NX7STuE9SImoIZrYSeBOwu8TPNgObAc5fuHDGa5c9+yyHgZfmNjZ9m3b8eOlj1NyDHUtDJ06UTheZFT6PIGgt6+lpfLfTwcHSF/5SgSZKu6AmhlpO26cZ+fmwAN1rvfRacLmc8in6rK/h8SVxn6TYA4KZDQJfAT7m7q8W/9zdtwHbANatWDHjCjxncpILDhxo3oBmy9NHc/lpo9ZS6TLNys+38sSzJJ6mFmtAMLM5BMHgPnf/apxjiV2jd/GaBYhMOzh+kHHGcbxgIVmt+flWnniWxNPU4uwyMuBeYL+73xXXOBKj0bt4zQKSTS2nbXNs8hinOT1dBA47g/q8j1Fqz8+38sSzpJ2mFucM4Srgg8DjZvbD3GO3u/s3YxyT7rRFUi7s3olynAkmmG/zYxpVOsTZZfR/geQdGpDkO20FK5GKRn2UPvoYY2x6lgDEnp9Pg9iLyh2l1XsGJTlYSVlqOW2fARtgnHEGGChoFZ1rc1uSnknaauNGKCA0U1JXC9cTqLQhXtN1Y8tpHBfLsHsnQ4az7Kzp7p2L5lzU9M9K4mrjRiggpFUtF+x6AlVSg5ukRlwXy3Z27yRxtXEjFBDSqtkX7OIAc/x4sPiupweWLq3vPaWrxXmxrLd7p9YZTRJXGzdCAaFTHD0a7IzqXngQT7UF5+IAEwaDUuc5i1QhbRfLemY0SVxt3AgFhE4xNZW/gEcv7I2keCYmggBz5Ej+MfdgNhFNS6neICW042JZT41ittfUM6NJ4mrjRiggNFOntYWGezVFz12Ympr5e1S9QUqY7WK5wBawb3Rfw/n9eu7oy72mnhlNElcbN0IBoZmSejdcT6CKpouiaaNocJCqfHpVFhYsiHsYbVfqYrnAFnA0e7QpheZ67ujLvabeGU3SVhs3QgGh1VqVTim+yLvn00bF6vmcpUvzqaLzziv8WaXdVWWGG9aujXsILTVbGqb4YrlvdF/TCs313NGXe83qOas7Kv1TDwWEVmtVOqX4Il/uNLXZRIPViRP5sxbM4Jxz8udDhwXrUFi4Vo1AqC11E70gT/ok4z7OFFOc9tPsPrObKaaqTrvUc0df7jWdlv6phwJCp6gnLRQNVtFgEm71HQaZI0dm1hEGB1UjEKC21E14QXZ3Rn20YGuJU34qWF1MdWmkegq6lV7TSemfeiggdIpW3KmHQSZMR4WK01KdVkyXmtSSugkvyOMeHCLleMFrJphgrs2tKo1Uzx29ZgHlKSB0mmbWLMLnV0pHKW3U1WpJ3YQX3h+P/xgIgsAUUwVbVYePV7NeoZ47+m6fBZSjgNBp1AKaKJ9eVd9Z3GlSa+pmUe8i5k/OD4KI9XA6e7ogEEC6F3elmQJCqymd0vVuuOqquIfQUvWkYaJBpJdexgl2JR1ggCmf6rrunqRQQGi1VqdTilNEr7wS/ILKawaqDVbNDmpa2dxxak3DFAQRRjnLz8LMmCI4wF55/XgoINQjSRe04hTRK68EbaPuhQFhcnLma6sda7N/T0prCcrlJ5ECQj10QRNpqlr3JGrXOQuddPhNNRQQOlG4B1G0VdSSd1qpCNS+J1G7zlnotMNvqqGA0Gl6evL7EEW3nNB2E5JQte5J1K5zFjrt8JtqKCB0ixMnCs9JCKmQ2zJ/OKeXfsbjHkbi1bonUbvOWUjbeQ7NoICQdqU6gCYngxRR8awgKXWPWrqWklTAr8O6K1fGPYTEq3VPonYdStNph99UQwGhHklaW1DtRbHU7CAutVzIVcDveLUubAufP5WdYoIJsmTJkGFJZkms46pG0ovUCgj1SMGdqcSvf4XSRdWodWHbot5FvDr1KoemDuE4Pbl/jmaPMn9yPot6FzXlwtvsfY/SUKRWQBBpofN6z6v8JKl5TcJxP86ADUwXfAGmfIpDk4cAmnbhbeZaiTQUqTNxD0CkW7mX/15KOzZ5jFezr3LGz3A6e5qJ7ASQL/hGL7xmRo/1kCEzHSziMuqj03s1hZJWpNYMoROUK7xC8LPoATiQPwRHeyrF4h8+/xpOjfTwm7cem15Y/sW7F3H24BTv+6//EffwEitMu4SyZBljDLKQsQwDNpDY7qA0FKkVEDpBpcLr4ODMn4+M5IvjxQXnJHXwJKmAX4Pf/9E8zl11ouTP3OHUSA/fevAcAH7z1mN88e5FfOvBc3jXr53APZnrCJNQEA3v/vutf/qAHccZZ5w+gj2QDk0eSuSFtxVF6mZTQOhm7ejgabRtNCmBqQ6r1i0o+bhZEAQAvvXgOdOB4V2/dmJ6xpA0SSmIhnf/Zhacrubj01tnr56zenosSbzwpuFwnlgDgpm9E7gb6AE+7+6fiXM80gJd1Da67c6Xp79ew8sc/NAFHOSJ6cduWLt2+uswKITBAEhsMIDkFESjaZde66XXepnyYIfUcBzVXHjjmu0kfUO/2AKCmfUAfwlcAxwG9pjZN9z9ybjGJFKTe+8FYNuLv5J/LJJ+W7I3//CL6x7g60/kg8NFF8G//MXFBW/3xbsXJTYoJCUvX23apdyFNymznSSKc4awATjo7s8CmNkDwA2AAoIk0/Aw276/NvLAr1S94G/J3pumv/7ud+FrT8HRM69w+S/9hF/8wE/4f1+6mG98+UIgmUEhKQXR6N3/qewpHMew6Q6iai7oSZntJFGcAeE84PnI94eBK2MaS7pVKrxGf3biBNNVS/fgewg2xFu6tPVjTaF8KmgtLFkCH/pQQ+/39rcHf/wXji1k0yULscfexMWrHmfy1As8ffIk33jyGebn/tNdff7a8m/WJkkqiEbrBJncP7Xc5SdltpNEiS8qm9lmYDPA+QsXxjyahKql8LplSz6nf/Rofovsycn83kcJ7+BpqeFhePrpwjTQVVfBxo1N/ZiNGynoJrp0zmWsuSL3/d71DA/DJbcVpplWrYIFPQtiWeyWtIJoI3f5SZntJFGcAeEIsDzy/bLcYwXcfRuwDWDdihVautNM0RnByAjceWfzPyMNbaP33hsJAGvhqs3woeYGgFKK00LR7zduBCJppv2Tj/PyI3Dum59gL8F6kvnz2juDSFJBtJG7/CTNdpImzoCwB3i9mV1AEAhuAn4jxvFIKyS0bTTaEVRLLSAua3ovC77Ye9n0Y8WF6nYHiDg1cpeftNlOksQWENx90sx+D/hXgrbTL7j7ExVeJlKf4oJwE2oBcYsWqoeH4YIPPsTXTxb+L7Tu4nhSTK3W6F1+kmY7SRJrDcHdvwl8M84xSOcqmAUsuRquen3TawFJsXEj8LN3ws/yjz2/4iH2crwgxbT65zojQOguvzUSX1SWJktDTr9eBbWAnISnglppeVGAGB6GE7//AHstv6fVRRfBGwbSmWbSXX7zKSB0m4Tm9Os1oxbQgo6gTvJvn7mJTZuCAvaTE4/z/b+dpO/sUdb/6jNAd9UhZCYFBEmVwgBAR9QC2sEdxsbg0UeD7zdtgiPfuYwnH4UNG2DxnvWYdXehWsoEBDObD2whaAf9lrv/XeRnf+Xu/70N45NuN2N1MF2dBqqXWRAEIAgKYWDYsIHpGQNULlSvuzjYsK8T6hAyU7kZwt8ATwNfAW42sxuB33D3MeDN7RicdKfCWUBzVgd3q+jiNzMYGsoHAygMBsWKC9XDwzD2wYfoX5wvVEPhpn2SbuUCwkXufmPu6380s63ALjN7TxvGJd1ExeCqFZ+VUO7shOHhIE0UXvSzWbjnnmANYrhYfceO8kEhqlQnU6kU05LXwtp+BYk0KhcQ+s0s4+5ZAHf/EzM7AgwDJfYzFqmeisG1K77AuwcX9P7+mX90xTWDoaEgGDz1FFxyCdxyC+zcWVhTqGdDvWiKCYIV1a+++YlZt/2WZCsXEP4JeAewI3zA3f+PmR0F/rzVA5MOUzwLUBqoJqWKwjt2BN9v2DBzplCqZjAykg8GmUz+5/39lYNBtTOTNb2XFaymfn7FQzz4nSP0L86nmFSoTi7zFJ3svW7FCt+7dWvcw5BqqBjcdOGMIFoDKC4Kl3rNHXfkv9+yJQgG0Z9XCga1zEwqCTfti1KhuvUuH7j8MXdfV+l5ajuVpmr2VtGSF971V1sUDi/cUTt3Fr6mmplBLTOTSoo37VOhOlkUEKR+bdoqWgKlLvCzFYWjs4lwFhGdXVRbM6i2XbVeKlQniwKC1CamraK7Xa0XeLMgpRO9cNdSM4iqdWbSqGoK1Z26aV/cygYEMzuXYEvqS3IP7Qfud/eXZ3+VdJq0bRXdieq5wBcfwhO+ptYLeS0zk1YoVagu3rQPVKhuhnIrldcAuwi2p/4BYMB64HYze4e7P9WeIUrbdeBW0dWqpc+/3eq5wJc7hKcazUo9NdPyosVyMPN0ORWq61NuhvDHwK3u/mD0wdyK5T8Bbiz5KkmlglnAVZvhKrquFtDMbppWafQCX8/nNSv11ArT/12KTpf716LT5VatUg2iGuUCwmXu/r7iB939K2Z2R6kXSIpodXCBZnfTdJJmpZ7apfh0ueFh4LYHCmoQChCllQsIp+r8mSTRjHUB3bE6uNoUUKu7adKu3TOTZip1PjUUFqlBhWooHxCWmNltJR43YHGLxiNN1O2bxNWaAmp3N43Eo7hIDaUL1d1YpC4XEP43MNsxWp9vwVikUUoDTasnBRR3N43Ep7hQvfgXg1bXqG5IM80aENz9U7P9zMw+1prhSK3StElcOzt4ak0BJbGbRuIR/O9TOIsolWbqxABR78K024DPNXMgUqWUzgLi6OCpJQXUrm6aJLe1yuyK00ydWqiuNyDor3AbpX1hWFwdPLWmgFrdTZOGtlapTqk9mYoDBKSvUF1vQEjPFqlp1GFbRcfRwVNvCqhV3TSVgmI2W/supJIcxQEC0lmoLrdS+SSlL/wGnNWyEXWjGWmg9M0CKml3B0+zUkDNSvGUC4r9/YW7kGrm0BkqFapXrQr+naQ0U7mi8mwdRtIk+VRQsovBzRBHB0+jKaBmp3hKBcWhoZknl2lBXOcpLlQPD8PLv/g45ybsdDntdtoupQ6M6fAgEIqzg6feFFAr6h6znU8wNBR8rQVx3aNUJ1MStv1WQGil4q2iOywNVK160zdxduRUqnsUqzS2SkFxaEgL4rpd8bbf4ely7dz2WwGhydLeEdQqtaZvktCRM1vd43vfq31sYVBcv74wKLpDX18wU4jSgjgpteXG9186Qv/iwllEMwvVCgiN6rCOoFaqNn0TZ5tq9H2z2ZkX6u3bg3/v2dOcsbnDU0/BsWNaECflrem9DH52WdnT5VatggU99c8iYgkIZvZZ4N3AOPAM8Dvufrz8q5KjcKvoz8Lr6YpaQLvE0aZaPCPJZuGee+DwYXjHOwov1OvXB79qGVsY5Pbsyf/+duyAvXth0aKZMwdIxvbSkmxLitZChIXqes+njmuGsB3Y4u6TZvY/gC3AJ2MaS0WFaSA0C2iD4nSNe+EFt5FZQqmZQPGMZOfOIBgsWxbk94sv1G97W36WEL6mUi0kfP3u3aVrEmnZXlqSqVSh+vkVDxXMICqJJSC4+7cj3z4CzDh3IValOoJUC2iraEfOyEhw0d6+Ha65Jnis3npCqdrEzp1BHn/DhsK7/ne8IwgG4YKx6EW92hba4s8bGgqem8nA4GD1C+RE6pFfC/H+qp6fhBrCzcCX4x7EjK2iu6QlNImiHTnr1weP7dpVmM/fs6f2nH2l2kS1nT7VttAWf97QUJCGOn0a5s7N/z41G5CkaFlAMLMdwNISP9rq7l/PPWcrMAncV+Z9NgObAc5fuLB5A0zpJnGdoFI7aXGbamjXLnj44eDOupqcfanPmK02ES4Qiyp1sa6lhbb483buhFOn4JJL4JZbZi5IU1CQuLUsILh7iW7tPDP7beB6YMjdZ90byd23AdsA1q1Y0dAeSmnaKrpTVdtOWtymes011efsK31GudXC5e76w/HU0kIbrYWYwdlnB8Egk1HxWJInri6jdwKfAN7u7qdb9TkqBidLre2k0dRLtTn7ajaRK54JRGsI4XvOmQMLFuQv1sVBpZYW2nDsg4P5z4vOLpIWDLRFd/eKq4bwF0A/sN2Cv2mPuPtHGn7X4WGAwoKw0kCJUU87aa3bXlSTFip+r127gm6ij3wk33L6xBPw/POwevXMMVR7gUzjoTtJWBAo8Ymry2hV096seHuIJUtgi2YBSVXrrqf1bHtR7jOK32toCA4cCFpMd+3Kt5wePw7Ll5duEa32It6sHVfbJa4FgZIcViZ9nzjrVqzwvUuXqhicYtG75lA1F9pa0hiVPmO2FcnFzx8agjvvzD92++31XRBbmYJp9nvX+99Hku3DH7bH3H1dpecloe20ai8dnWJbRsXgtGokhVJrzr7UZ7gHxenixW1hgbdUoTlqx47CdQnR9yin3h1XK2lFeqfWGZx0llQFBJYu1WwgxdqRQpntM555Bp59Nv+8bDa4eA4MBKuOi4vW99wD//EfcOWVhbWGAwfyXUK1XoCbeTffqvROHOdWSHKkKyBI6rX63OJSn/G978GLLwYrnrdvDz7vr/4KDh4M1gSMjuYXuhUXmsNtK6K1hrBLqJYLcKsO24Hm7feUxiK4NJcCgrRdq1Iopd7TPbjgnzoF4+PBBW7HjiA49PXBhReWLjRD8Hg4E8hkCheT1XIBLt4rqbjbqdFjOZuV3klbEVyaTwFBOppZfv+jnTvhxIng60wGrrsuX1MIL8rhnXxYK6i0qK3chnvu+bMTwiCze3d+L6NwF9VG0kbNTu+0YwYnyZWp/BSR5ClujivXLBe90w1lMjOfE83L79xZmEIZG8vXHaJ27AieNzyc/zocz/btsH9//v2GhuDkyWAvo2w2n46q9/cfTe/cfnt+c77oOOrRjhmcJJNmCJI6tebjs9mgZnDyZBAIenqCX+GF89prg/eJposqLWqLfu8e/Nq7N3h+mHvfsyfYnO+CC/JBIdzYbt68whXLpZQrQiu9I62ggCCpUmt3TRgsDh4MagbXXRc8Hm4098gjhWmjcBuLqOJFbeGRl9Faw+hocNBNqfqCe1CkPnUq2Mvojjsqb2xXTdBTekeaTQFBUqXW7hqzoLV0zZrgTj2sJ7gH+fwzZwq7hnbvhte8pvA9wrz8xo0zF7GFM4VwNnDsWP514Th37sxvbBfODKLBpNJ+TMWzk2y28IyG4t+vSL0UECR1au2u2bgxWGsITK5bAAALPUlEQVQQvhaCNNE11xR2DbkHweD48dnbLqO7lEYDUnhuQ1T0/OXwsJ3iYFJcy4j+/iCYWezYEQSSK6/Mv4f2FpJWUFFZUme27ppKheXiTfCiF/fwscsum5mX37Ch8E6+uEgdfm64lmHLluDfe/bAc8/BunWFwSR8v1LBIDqWoaFgNnD6dFD/iAaUsbHGCscipWiGIKlSzeIpqG5FcKnAEraIlsvLF7/OLLjwr18fXOjDFBME9YaxsaD9NNw2u5o8f1jPmDcv+P706aCTaN487S0kraMZgqTKbN014V339743s/1zx47pndGnlWvbDFtOo59Z6XXHjgU/278/n+YZGgqCwa5dwePhe1YTDMLPuPLKYIO9s88OitLhTEHBQFpBMwRJndm6a2DmbGG2DqR62zbLva6vLyhcP/tsEAT27Aku4O7B49WKfkaYJgpnCmaV21VF6qWAIKk0W3fNbAXfUiuK623bLPe6cBbwT/8U3NEDvPvdhbusVqO4o6l47UP4e1VQkGZSykg6SnHBd2Sk8OfFKaR62zbb0e6ZyRTORoqL0goG0myaIUhHKS74hsVZCO7SW3kCWLhdxa5d+dXIJ08G34efX+vnafGZtJMCgnSMUh1I4QU6TLWYtbZL57nngnEMDQUBYPv24LOfe67+99TiM2kXBQTpGKUKvuHK5IcfLqwzNOuiWnz3Hl0RHf38gQFdyCX5FBCkoxSnWEKDg/mvm3UCWKn9hsbGZi5iqydVJBIHFZWl40S7icJdR5u9RXR0v6HwvcJ01fj47OsYRJJMMwTpWK3eIjra4rp7d+vrE1G1ns/czPOcpXMpIEjq1HJxa0WXTjRVtGlTEAxefTU4YyGaPmrVBbfW8yCafZ6zdC6ljCRVSp1MVmpriqhmdulEU0Xbtwe/jh0L1jtMTQXfhyerlRtTMz6/OFVVasO7Wp8v3U0zBEmNWg/HaYVwhhEeejMyAr29wZqD5cuDsTz9dH4L7WaPqZ7zIGp5vnQ3zRAkNaIb2T36aHDyWHTNQbsubtHOocFBWLw4WHdw4kQQIA4fnjmmWs6Arubzi8+ILvf7r/X50r0UECRVknBxC9Mug4P5TefCx+fNCx6PjqmeNFc1nx9VrmuqnvMjpDspIEiqxH1xK7X99fr1Qfoo3Nk0OqZm5/DLbdtd6s+h1udLd1MNQVKjmsNx2lFDiLayRsd24YWwefPMMTUzh19rK22rW2+ls8QaEMzs48CfAovd/Vil50t3S8rFrbiVdWAgv3fRbGOq5QzoWj+/UiutNsiTasUWEMxsOXAtcCiuMUj6JOXiFv28SmOaLc3VyLhrbaXVBnlSjThnCH8GfAL4eoxjkBRK4sVttjElIc0lUq1YAoKZ3QAccfcfWYX/G8xsM7AZYOHC89swOpHmSUqaS6QaLQsIZrYDWFriR1uB2wnSRRW5+zZgG8CKFevUEyGpk5Q0l0glLQsI7r6p1ONmdhlwARDODpYB+8xsg7sfbdV4ROKUxDSXSLG2p4zc/XFgSfi9mf0UWKcuIxGReGlhmoiIAAlYmObuK+Meg4iIaIYgIiI5CggiIgIoIIiISI4CgoiIAAoIIiKSo4AgIiKAAoKIiOQoIIiICKCAICIiOQoIIiICKCCIiEiOAoKIiAAKCCIikqOAICIigAKCiIjkKCCIiAiggCAiIjkKCCIiAiggiIhIjgKCiIgACggiIpKjgCBSJffy34uknQKCSBWGh2HHjnwQcA++Hx6Od1wizaSAIFKBO4yNwaOP5oPCjh3B92NjmilI5+iNewAiSWcGmzYFXz/6aPALYMOG4HGz+MYm0kyaIYhUIRoUQgoG0mkUEESqEKaJoqI1BZFOoJSRSAXRmkGYJgq/B80UpHMoIIhUYAb9/YU1gzB91N+vYCCdI7aAYGYfBX4XmAL+xd0/EddYRCrZuDGYKYQX/zAoKBhIJ4klIJjZ1cANwBvdfczMlsQxDpFaFF/8FQyk08RVVL4F+Iy7jwG4+4sxjUNERHLiCgirgbeZ2W4z+66ZrY9pHCIiktOylJGZ7QCWlvjR1tznLgTeDKwHHjSzC91nNvGZ2WZgM8DChee3argiIl2vZQHB3TfN9jMzuwX4ai4APGpmWWAR8FKJ99kGbANYsWKdur5FRFokrpTRPwJXA5jZaqAPOBbTWEREhPjaTr8AfMHMfgyMA79VKl0kIiLtE0tAcPdx4ANxfLaIiJSmvYxERARQQBARkRwFBBERARQQREQkRwFBREQABQQREcmxNLX/m9lLwM9a8NaLSPfCOI0/Xmkef5rHDhp/tVa4++JKT0pVQGgVM9vr7uviHke9NP54pXn8aR47aPzNppSRiIgACggiIpKjgBDYFvcAGqTxxyvN40/z2EHjbyrVEEREBNAMQUREchQQIszso2b2lJk9YWb/M+7x1MPMPm5mbmaL4h5LLczss7k/+38zs6+Z2YK4x1SJmb3TzH5iZgfN7A/iHk8tzGy5mT1sZk/m/r7fGveYamVmPWb2AzP757jHUiszW2Bm/5D7O7/fzN4S95hAAWGamV0N3AC80d3XAn8a85BqZmbLgWuBQ3GPpQ7bgTe4++XAAWBLzOMpy8x6gL8E3gVcCrzfzC6Nd1Q1mQQ+7u6XEhxl+7spGz/ArcD+uAdRp7uBh9z9EuCNJOT3oYCQdwvwGXcfA3D3F2MeTz3+DPgEkLrCkLt/290nc98+AiyLczxV2AAcdPdnc+d7PEBwQ5EK7v6Cu+/LfX2S4IJ0Xryjqp6ZLQN+Gfh83GOplZmdA2wE7oXgfBh3Px7vqAIKCHmrgbeZ2W4z+66ZrY97QLUwsxuAI+7+o7jH0gQ3A9+KexAVnAc8H/n+MCm6oEaZ2UrgTcDueEdSk88R3Pxk4x5IHS4gOD/+b3Ipr8+b2dlxDwriO0IzFma2A1ha4kdbCf4sFhJMn9cDD5rZhUk62rPC+G8nSBclVrnxu/vXc8/ZSpDOuK+dY+tWZjYIfAX4mLu/Gvd4qmFm1wMvuvtjZvaf4h5PHXqBXwA+6u67zexu4A+AP4x3WF0WENx902w/M7NbgK/mAsCjZpYl2GfkpXaNr5LZxm9mlxHcdfzIzCBIt+wzsw3ufrSNQyyr3J8/gJn9NnA9MJSkQDyLI8DyyPfLco+lhpnNIQgG97n7V+MeTw2uAt5jZtcBA8B8M/uSu6flWN7DwGF3D2dk/0AQEGKnlFHePwJXA5jZaqCPlGya5e6Pu/sSd1/p7isJ/sL9QpKCQSVm9k6CFMB73P103OOpwh7g9WZ2gZn1ATcB34h5TFWz4M7hXmC/u98V93hq4e5b3H1Z7u/6TcCuFAUDcv9fPm9mF+ceGgKejHFI07pqhlDBF4AvmNmPgXHgt1Jwl9pJ/gLoB7bnZjmPuPtH4h3S7Nx90sx+D/hXoAf4grs/EfOwanEV8EHgcTP7Ye6x2939mzGOqZt8FLgvdzPxLPA7MY8H0EplERHJUcpIREQABQQREclRQBAREUABQUREchQQREQEUEAQKcvMRko89kdmdsTMfmhmT5vZV2fbGM7M/nNuN9GsmSXm7FyRUhQQROrzZ+7+8+7+euDLwC4zW1zieT8GfhUYbuvoROqggCDSIHf/MvBt4DdK/Gy/u/+k/aMSqZ0Cgkhz7AMuiXsQIo1QQBBpDot7ACKNUkAQaY43kZBTr0TqpYAg0iAzu5HgLIr74x6LSCO0uZ1IGblzMf498tBdwHzgvxGclXE2QSfRVnefsYWxmb0X+HNgMXAc+KG7/1Krxy1SDwUEEREBlDISEZEcBQQREQEUEEREJEcBQUREAAUEERHJUUAQERFAAUFERHIUEEREBID/D/KQ9Gj7Au5AAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff3a784a450>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_decision_regions(X_train_lda, y_train, classifier=lr)\n",
    "plt.xlabel('LD 1')\n",
    "plt.ylabel('LD 2')\n",
    "plt.legend(loc='upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以发现，有一个样本被错分类。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果降低正则项的影响，完全正确分类训练集。当然了，过拟合并没有什么好处。我们看一下现在模型对测试集的分类效果："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_test_lda = lda.transform(X_test_std)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAH6xJREFUeJzt3X9wXeV95/H3V5ItxZGMsWWHBP9kjfGPYJKgGKdOTB0bllKCkzq7Q7JNS2DWCdswUDKT1Hgys5mk0G13STJtl6kXpzPdspC0hKbtpBQb02iWDrb5uY4xMcQBgwPY8tQgYUuypO/+ce61rq6uru7V/fGcc8/nxXike3Wl+/UPzuec5/s8zzF3R0REpCl0ASIiEg8KBBERARQIIiKSoUAQERFAgSAiIhkKBBERARQIIiKSoUAQERFAgSAiIhktoQsoR2d7uy+eMyd0GSK119PDG/M7mdXeHroSaQAvPPNCj7vPnex1iQqExXPm8NT27aHLEKm5HXef5PgDS9i8alXoUqQBrG5b/Wopr9OQkYiIAAoEERHJUCCIxNDxneeHLkFSKFE9hELOtrTw+kUX0T9jRuhSJtR2+jTzjxxh2tBQ6FIkQdQ/kHpLfCC8ftFFdCxYwOKODswsdDnjuDsne3t5HVhy+HDockREJpT4IaP+GTOYE9MwADAz5nR0xPoKRkQEGiAQgNiGQVbc65OY6e4OXYGkVEMEgkhDeeklaGsLXYWkkAKhCm667TbmrVzJB9evD12KiMiUKRCq4MYbbuCRBx8MXYY0iG/f/cXQJUhKJX6WUVmuvhp6esY/39kJjz465R+7/mMf45WjRysoTGSszZdfHroESaF0BUJPDxTaHK9QSIiIpEzwISMzazazZ83sH0PXIiKSZsEDAbgNOBS6CBGRtAsaCGY2H/hN4L6QdYiISPgrhO8CXwNGAtdRkc996Ut87Npr+fnLLzP/ssvYef/9oUsSESlbsKaymV0HHHf3p83s14u8biuwFWDh7NmVvWln59gG8vBw9PH88+HYsdHnm5rg/e8v+cc+8Bd/UVldIhk77j4J2ulUAgk5y2gdcL2ZXQu0ATPN7K/d/bdzX+TuO4AdAF2LFnlF75g/tfTYsejgn28k0RcskmDHd57P5nXrQpchKRVsyMjdt7n7fHdfDNwA7MkPAxERqZ/QPQQREYmJWCxMc/d/Af4lcBkiIqmmKwQREQFicoUQTFNT4QZyoUaziEiDS3cglDG1tJjXjh3jd77yFd46cQIzY+sXvsBtW7dW5WeLiNRL6gLBHXJvYJb/eCpaWlr4H9/8Jh9ZvZrevj4u37SJq668kpWXXFLZD5ZU+fZSTXeWsFI1NrLjr9q459734JnVDO5wz73vYcdfVXZ3qve/7318ZPVqADra21mxbBnH3nij0nIlhbQGQUJKTSC4Q2+f8cDDredC4Z5738MDD7fS22fnQqJSrxw9yrMHDnCF9rMXkYRJzZCRGdxxyxkAHni4lQcebgXgc58Z4I5bzlQ8bATQ19fHlptu4rvf+hYzOzoq/4EiInWUmisEGBsKWdUKg7Nnz7Llppv4T1u28FvXXVf5DxQRqbNUBUJ2mChXbk9h6j/Xufn221mxbBl33HJLZT9MRCSQ1AwZ5fYMssNE2cdQ2ZXCE3v38r//5m+4dMUKPrRhAwB3bd/OtZs2Ff6Gu+6C3t7xz3d0wJ13Tq0IEZEKpSYQzKCj3cf0DLLDRx3tXtGw0cfXrsWPHy/9G3p7ob298PMiIoGkJhAAtv5O/5h1B9lQqEYPQaQS35jWQuusGaHLkJRLVQ8Bxi9CUxhIXHRdMit0CZJyqQsEEREpLFVDRomghnMqtS4aDF2CiAIhiI6OiQ/6ajin1oUtF4YuQVJOgRBCsTP9bdvqV4eISA4FQhX09/ezfvNmBgYGGBoe5rPXXcc3v/710GVVj4axRFJBgVAFra2t7HnoIdrb2zl79iwf/9Sn+I2NG1nb1RW6tOrQMFZN/f7zHcxZ+nboMkTSFwiv2K942g7xjvUx09u53Few2D9Q0c80M9ozB8yzZ89y9uxZLK7zWXW2H0tLuzTlVMJL1bTTV+xXPN60n3ftDK1M5107w+NN+3nFflXxzx4eHuZDGzYwb+VKrrryyqlvf93RAX19439Va/fU7Nl+/i+d7YukXqquEJ62QzTTzLTMbzv78Wk7VPFVQnNzM889/jin3n6bz9x4Iz87dIgPrlhR/g/SWbpIqvQM9XB06Cj93k+btbGwZSGdLZ1BaknVFcI71kcLzWOea6GZd6yvau8x67zz2LBuHY/s2VO1nymNbc4W9Q/Sqmeoh8NnDzPog7TQwqAPcvjsYXqGeoLUk6pAmOntDDE85rkhhpnpBRqmZTjR08Opt6P/qc+cOcOun/6U5RdfXNHPjJVaD2MJq1pXhS5BAjg6dJQmmmi2ZsyMZmumiSaODh0NUk+qhowu9xU8bvuB6MpgiGGGGeZyn8LQTo433nqL3731VoaHhxlx5z9efz3XXX11NUqOBw1jidREv/fTkncYbqKJfu8PUk+qAmGxf4ANIx+t+iyj1atW8WxShoiKrZIWkbpqszYGfZDmnKHsEUZos7Yg9aQqECAKhUoDINF0th8rO+4+CTvPD12GBLKwZSGHzx4Gj64MRjL/LWxZGKSeVPUQROKmu/dDzFygNQhp1dnSybJpy5hu0xliiOk2nWXTlgWbZdQQVwjuHt+FYET1iYgU0tnSGSwA8iU+ENpOn+Zkby9zOjpiGQruzsneXtpOnw5disTQ8h++CugKoZbiNM8/7hIfCPOPHOF14MSM+N5+sO30aeYfOTK1b9ZWEw1vw0JNOa2V7Dz/JprGzPMHFAoFJD4Qpg0NseTw4dBl1I42lhOZstx5/kA0m8ej57OB0DPUwy/O/oLTfhrDaKONpdOXpjIwEh8IIpJctR7OmWyef89QDy8OvsgggxiG45zhDIcGD7GCFakLhWCzjMxsgZk9bmYvmNlBM7stVC0iQezcGbqCoOqxbUObtTHCyJjncuf5Hx06yhBDNNGEZf4DGGY42GrhkEJOOx0CvuruK4G1wO+Z2cqA9YjUX1uYBUhxUI9tGxa2LGSEEYZ9GHdn2IfHzPPv9/5xgQHgeLDVwiEFGzJy9zeANzKf95rZIeBC4IVQNQWhprGkVKHhnBEf4W1/m389869VGULKfu9Ew1LZlcL5DAu2WjikWPQQzGwx8GFgb9hKApisaaytJhrWN778JVpDFxFQ/rYNQz5EP/1VnxFUbJ7/wpaF43oIEF0hDI4M0jPUk6o+QvBAMLN24CHgdnd/p8DXtwJbARbOnl3n6mJAVwkNq3XRIJsvXxe6jAnVuuGbv23DgA9gGNOZHg0hFZgRVG2dLZ0sZ/m5WUZZrbSCkbopqkEDwcymEYXB/e7+o0KvcfcdwA6ArkWLtOS3XBqSkimox/z9/OEcgOlMZ1rTtHOvqcfOn9kriGf6n4muWCznnik1DqS4CRYIFi0r3gkccvd7QtXRkHJD4NQpyK7gbm6GCy6AN9+Mnt+2bez3KSQko5T5+9WQO5yTPSDnqufOn3HbijqEkFcI64AvAAfM7LnMc3e6+08C1tQYcvsSb78dBQHA8PDoR7PxvQstdpOMEAfH0Dt/lrIVdaNvgxFyltH/BeK3+VC9qWmcTt3dcMmVoauYUIh9+iebEVRrkwVSGrbBCN5UTj0N0UgMhTpbD7nz52SBVK9htJAUCCIyTuiz9VCKBVIaegwKhEbX3DzaO3CHvr7oY4v+6kP69k0fD13CpOK0T38cxO12l7WgO6Y1oo6O6MDf1xc1js87L/q1YAHcfXf0sb199DXZX+pb1NXmtWtDlyBlmGwbjEag08RGVKwvUcm6BK1pkBRLwzCaAiFtKrm/gu7NICnX6MNoCgSZWP4VwalTo+saLrggXF0iUhMKBJlY/hVBNgyyTWoRaShqKouICKBAEKm/lN8pTeJLQ0ZpU8lWGdnhoux6hnK+V8759t1fpOuKxaHLEBlHgZA2lUwPzTaS+/qi9QzlKHfKaoNPcb2w5cLQJSROo28sFwcKBJlYNTfeK3fKqqa4So4kbCzXCIGlQJCJNcCZuDSGuG8sl4TAKoWayiISe/0e3Ws5V5w2lssNLDOj2ZppoomjQ0dDl1YWBYKIxF6btTHCyJjn4rSxXNwDq1QaMhKRWOsZ6mFwZJDTnKbJm5jOdJqsKVYbyzXKTqgKBKmPchvUDXonuW9Ma6GVwclfKEDO2Lw10eqtnOUs/fTzXt7LsmnLYjM+H/r2n9WiQJD6KLdB3aAN7dZFg2xety50GUGVMxsnd2y+2ZqZznSGfZhpNi02YQCNsxOqAkFE6qbc2ThJuktZI+yEqqayiNRNubNx4t5MbjQKBBGpm3Jn46ThLmVxoiEjEambcmfjNMrYfFIoEESkbqYyG6cRxuaTQoEgInWjM/54UyBIMiVwN9Tff76DOR8cCF1GcDrjjy8FgiRTQndD7bpM96KW+NIsIxERARQIInUzZ8vboUsQKUqBIFJHulOaxJkCQUREADWVJakadDdUkZAUCGlU6pTNOE/tDP3+peruZscTq6LPd57Pjw8epOuSWRo6klgKGghmdg3wPaAZuM/d/yhkPalR6pTNhE7tjIMdd5/MfLYK1q2D9euZ9zIcGjrAUxzkKU4BMLMDNixcFa5QkRzBAsHMmoE/B64CXgf2m9nfu/sLoWoSmbLcK4GsbdvGvWxFy6Xw1KXZb2Hurx3gnbUHx7xGVxASyoSBYGYzgW3AfOCf3P3/5Hztf7r7f6nwvdcAL7v7kczPfBDYDCgQJBl27mTH8U9nHqyCefPg5ptL/vb16wFGAwLgtUWP8BSnzl1BLF0Kq1p1BSH1UewK4S+Bl4CHgJvMbAvweXcfANZW4b0vBF7Lefw6cEX+i8xsK7AVYOHs2VV4W5EKjLkS+PS54aBqWfDqNfBq9PmhoQPAQV5m9ApCASG1VCwQ/p27b8l8/ndmth3YY2bX16Guc9x9B7ADoGvRIq/ne4sAeSGwquBQUC3kDi9lvcyD5wJi6VKY1azhJameYoHQamZN7j4C4O5/aGbHgG6gQKexbMeABTmP52eek1ordcpmiqd2jjaFgXkbYN3FVb0SmKp5T91w7vODJx6hde4pNailasy98Em3mf0x8Ki77857/hrgT9394ore2KwFOAxsJAqC/URDUgcn+p6uRYv8qe3bK3lbkQmNCQGo25VANXR3w/I7Hhzz3OZVCgeJrG5b/bS7d032ugmvENz9axM8/whQURhkfs6QmX0F+GeiaaffLxYGIlU3pilM2U3hOFm/Hsi5ejg0dIAfnjhG69xT555T/0EmE3Qdgrv/BPhJyBokfUavBKrfFI6LFS2XwquXFm1Q6wpC8mmlskxNnFcx58tfI5DgK4Gpym9QH+96kB8fHA2HrktmAdp8L+0UCDI1MV/FPLYfUP4agUaX25x+bdEjPHGCcw3qmR0w730aXkqjooFgZnOAzwPLM08dAh5w95MTf5dIIPk9gQQ1hUNa8Oo10SeZ4aVsg1rDS+lTbKXyCmAPUdP3WcCAjwJ3mtkn3f3F+pQoMoHubnjppZwQ+LRCoAryG9T5w0tqTjeuYlcI3wJuc/cf5j6ZWbH8h8CWgt8lUkv5i8TWbYWbG68pHCfz8mYvnXwSXs7svzQzsyRF6x8aQ7FAuNTdP5v/pLs/ZGZ31bAmkbECrRSW8Va0ZBrTORv0Lb9jfINazelkKhYI707xa5IGtV7FXOHGcYW4g9nEj6V8+cNL+ZvzqUGdLMUCYZ6Z3VHgeQPm1qgeSYpaTC3N3ziuilcC3d0wMACbNkUh4A67d0Nra0MuQwgmd3O+7u7oY26DWv2HeCsWCP8LmOh0774a1CJplH8lUIOFYu5RGOzbFz3etCkKg337YM0aXSnUyrm/xswVRHc3kDd7SQERL8W2rvjmRF8zs9trU46kwdiN47bWfOM4sygEIAqBbDCsWTN6xSC1V2h7jdzV09qcL7ypLky7A/huNQuRBhaDlcLZUMiGASgMQtPd4+JnqoGg/42kuBo0hSuR7Rnk2r07OaGQHdaa6GPS6e5x8TDVQNCNamScsdtFxGfjuGwYZHsGuT0EiH8oZBvi06fD4CBs3AiPPRY1xAcGGrcxPtnd47R6uvqKrVTupfCB34D31KwiSZb84aAYrhEwiw6auT2DbE+htTXeYZDbEJ81C/7t3+DwYTh1avTxFVc0zpXCRCbbnE93j6uOYk3lxr8tlkzJuI3jYnIlUMz69WMPmtlQiPtBNL8h3tcHb70FM2ZEv58rrkjG76PacldPA+x98gBz1h7U3eMqpN1OpSRJvptYVv5BMykH0dyGeEcHvPtu9LHeoRbnhX35DWqtnp4aBYIUlr9xnLaPDia3IZ5dHN7bG4VCvRrjSVrYV2h66xMnjtE6d3RxHKhBXYgCQUZp47iy1fqsObchPmtW9HjBgtEewt690etqGQpJX9iXe/e47OZ8c9Zq/UMhCoS008ZxU1aPs+bchnixWUa1PCA30sK+/M35oPDd49I6vKRASCHdTaxy9Txrzm2IZz/mhlA9DsiNvLAv/+5x+ZvzLftAegJCgZASjdAUjpN6nzXnzo4q9LHWkr6wr1S5ax8gugp8544HzwUENPb6BwVCI8tdLayrgKpr5LPmXElf2FeJQtt7N/Ld4xQIDabeG8elWVrOmpO8sK/a8ldPD5w4xstzRwMi6Q1qBULSxWDjuDQqdtbsDlddNXqgjPssnFIkdWFfLeXOXsrKbVAnsf+gQEiimG0cl0YTnTUfOQK//OXo6+I8X79cSV3YV0/5Dep3ek8lavW0AiEhxm0cp6ZwcPlnzQBLlsD+/aNDR0mary/VlX/3uPzV03HsPygQ4mzMlQAKgRjKPcCbjQ4VJX2+vlRXfnM6rnePUyDETFy3kJbSpGXmkVQmrnePUyCElt8UBl0JJFhaZh5JdeVv731oaOzd47oumQVQ8wa1AiGE/O0i1BRuCGmery/VlRsQry16hCdOQOvcqEE9swPmva82w0sKhHrRnkENT/P1pRYWvHpN9Eleg7oWd48z9+TcDbNr0SJ/avv20GWULr8prH5AKsT5vgHSeI53PTjmcaG7x61uW/20u3dN9rOCXCGY2Z8AnwIGgV8AX3T3U8W/KxnUFBbN15d6mpc3e+nkr43ePW5mmfe9DDVktAvY5u5DZvbfgG3A1wPVUhk1haUOdNUhpYjOPcffPa5UQQLB3R/Nefgk8NkQdVRi9EpATWGprSTdrUziZXR66+dKen0cmso3AT8IXUQpxm4cpxCQ2kv63cokWWoWCGa2G7igwJe2u/uPM6/ZDgwB9xf5OVuBrQALZ8+uQaVFaOM4CayR7lYm8VezQHD3TcW+bmY3AtcBG73IVCd33wHsgGiWUTVrLCh/4zg1hSUwrX6Wegk1y+ga4GvAle5+OkQN53R3w0sv5YSANo6TeNHqZ6mXUD2EPwNagV0W/Yt+0t2/XLd3HxMCq2DeBtimoaBGkMTZOMVq1upnqadQs4yWhnjfMTOD1m2FmzUU1EjiMBun3ECarGatfpZ6isMso9rRGoHUKDQbZ9eu6N4E2dk4UNsDaLmBVOoMIt2tTOql8QJBG8elUv5snD17ooPoJz85+nwtrxamMj20nBlEWv0s9dAwgTB2OEgzg+KsVuP8ubNx3OHdd0e/Vuu5+1OdHqoZRBInyQ2E3Omh8+Y1RAgksSFarlqO8+fOxunI7OHy2GPRwdas9nP3p3Jw1wwiiZNEBcKJN4dHrwTmbYV1Fyc+BLLi0BCttVquui00G2fXLviHf4i+3tFR+4NsuQd3zSCSuElUIAAN2RROy/YEtVx1mz8bJ2vGDGhqir5eyzPvqRzcNYNI4iZZgXBBoZ0wki9N2xPUcsw8OxsHooPx/v2wcWN04/tan3mXc3DPDfj162FkRDOIJB6SFQgNLC3NxULDKrt2RQft3MVYU/19Z7+vlINztXs2pUwPLTQ0+NhjY4cGG+3vXJKjKXQBEplo/DlBN7SbVP6wyp13QmdndEDctSv6evY13d2Vvdf69WMPxtmDc/ag29099s+3Wu9bbHpo7tBg9r2zfx4DA431dy3JpCuEGEhLc7HQOP+SJfDLX0a/oLq9k4kOzqF6NmkaGpRkUiDEQJqai/nDKlddFX3cvx/uuiv6vF7TQ6H+B+bJhgbTMPVY4ktDRjEx2RBHI8lfgZsNhax6nC3nhkI937fY0GCthrFESqVAiJE0bk8QqndSzvvmPzfV2gr1UNasiR7v2gX9/eovSFgaMpJgQvVOynnfai4YnGxo8BOfiJ5Tf0FCUSBIMKF6J6W+by2az5NNTU3D1GOJLwWCBBVqa+dS3reU5vNUQqHY7CftayQhqYcgwYXqnZTyvrmh0NcHvb3R6ufc4aNqNH0n6y/k9hAm+rzQY5FyKBBEisg9a89uqX3vvdF2E9Vs+k40jNXZObpGI7ee7m7NSpLqUyCITCD/rP2uu2D5cnjxxWiPxb17q9v0zZ96DNHCvZ6e8TOP+vs1K0mqTz0EkQkUOmu/5ZYoDLI7qFZ7fL/QGo2JZh5lX6NZSVItukIQKSL3rD27EV1HB7S3R1+v9ZqJYgvoQi2uk8alQBCZRG4Ded8+uOKKsU3fWoZCsQV0adgQUepLQ0YiJQixZqLYArrsQX///sbeEFHqS4EgUqJ6r5mYLIQgHRsiSv0oEETKUO81E5OFUIhFfdK41EMQibliIZTGDRGldhQIIiICKBBERCRDgSAiIoACQUREMhQIIiICKBBERCRDgSAiIkDgQDCzr5qZm1lnyDpERCRgIJjZAuBq4GioGkREZFTIK4TvAF8DtDejiEgMBAkEM9sMHHP350O8v4iIjFezze3MbDdwQYEvbQfuJBouKuXnbAW2AsyevbBq9YmIyFg1CwR331ToeTO7FFgCPG/RTlzzgWfMbI27v1ng5+wAdgAsWtSl4SURkRqp+/bX7n4AmJd9bGavAF3u3lPvWkREZJTWIYiICBCDG+S4++LQNYiIiK4QREQkQ4EgIiKAAkFERDIUCCJFuBd/LNJIFAgiE+juht27R0PAPXrc3R22LpFaUSCIFOAOAwOwb99oKOzeHT0eGNCVgjSm4NNOReLIDDZl1trv2xf9AlizJno+WmQv0lh0hSAygdxQyFIYSCNTIIhMIDtMlCu3pyDSaDRkJFJAbs8gO0yUfQy6UpDGpEAQKcAMWlvH9gyyw0etrQoDaUwKBJEJrF8fXSlkD/7ZUFAYSKNSD0GkiPyDv8JAGpkCQUREAAWCiIhkKBBERARQIIiISIYCQUREAAWCiIhkKBBERARQIIiISIYCQUREAAWCiIhkKBBERARQIIiISIYCQUREAAWCiIhkKBBERAQA8wTdINbMTgCvBnjrTqAnwPtWQ1JrT2rdkNzak1o3JLf2etW9yN3nTvaiRAVCKGb2lLt3ha5jKpJae1LrhuTWntS6Ibm1x61uDRmJiAigQBARkQwFQml2hC6gAkmtPal1Q3JrT2rdkNzaY1W3eggiIgLoCkFERDIUCGUws1vN7EUzO2hmfxy6nnKZ2VfNzM2sM3QtpTCzP8n8ef8/M3vYzGaFrqkYM7vGzH5uZi+b2R+ErqdUZrbAzB43sxcy/7ZvC11TOcys2cyeNbN/DF1LOcxslpn9bebf+CEz+1jomhQIJTKzDcBm4DJ3XwX898AllcXMFgBXA0dD11KGXcAH3X01cBjYFrieCZlZM/DnwG8AK4HPmdnKsFWVbAj4qruvBNYCv5eg2gFuAw6FLmIKvgc84u7LgcuIwe9BgVC6W4A/cvcBAHc/Hriecn0H+BqQmKaRuz/q7kOZh08C80PWM4k1wMvufsTdB4EHiU4gYs/d33D3ZzKf9xIdmC4MW1VpzGw+8JvAfaFrKYeZnQesB3YCuPugu58KW5UCoRzLgE+Y2V4z+6mZfTR0QaUys83AMXd/PnQtFbgJ+KfQRRRxIfBazuPXSchBNZeZLQY+DOwNW0nJvkt0ojMSupAyLQFOAH+ZGe66z8zeG7qoltAFxImZ7QYuKPCl7UR/VrOJLqk/CvzQzC7ymEzTmqT2O4mGi2KnWN3u/uPMa7YTDWvcX8/a0sbM2oGHgNvd/Z3Q9UzGzK4Djrv702b266HrKVML8BHgVnffa2bfA/4A+EbooiTD3TdN9DUzuwX4USYA9pnZCNE+JCfqVV8xE9VuZpcSnY08b2YQDbs8Y2Zr3P3NOpZYULE/cwAzuxG4DtgYl/CdwDFgQc7j+ZnnEsHMphGFwf3u/qPQ9ZRoHXC9mV0LtAEzzeyv3f23A9dViteB1909eyX2t0SBEJSGjEr3d8AGADNbBkwnAZtpufsBd5/n7ovdfTHRP8SPxCEMJmNm1xANB1zv7qdD1zOJ/cDFZrbEzKYDNwB/H7imklh0prATOOTu94Sup1Tuvs3d52f+Xd8A7ElIGJD5/+81M7sk89RG4IWAJQG6QijH94Hvm9nPgEHgd2N+xtoI/gxoBXZlrm6edPcvhy2pMHcfMrOvAP8MNAPfd/eDgcsq1TrgC8ABM3su89yd7v6TgDWlwa3A/ZkTiCPAFwPXo5XKIiIS0ZCRiIgACgQREclQIIiICKBAEBGRDAWCiIgACgSRosysr8Bz/9XMjpnZc2b2kpn9aKLN4MzsP2R2EB0xs9jcO1ekEAWCyNR8x90/5O4XAz8A9pjZ3AKv+xnwW0B3XasTmQIFgkiF3P0HwKPA5wt87ZC7/7z+VYmUT4EgUh3PAMtDFyFSCQWCSHVY6AJEKqVAEKmODxODO16JVEKBIFIhM9tCdL+JB0LXIlIJbW4nUkTmvhe/ynnqHmAm8J+J7oXxXqKZRNvdfdz2xWb2GeBPgbnAKeA5d//3ta5bZCoUCCIiAmjISEREMhQIIiICKBBERCRDgSAiIoACQUREMhQIIiICKBBERCRDgSAiIgD8f5RRIdiMEmLFAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff3a784a5d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_decision_regions(X_test_lda, y_test, classifier=lr)\n",
    "plt.xlabel('LD 1')\n",
    "plt.ylabel('LD 2')\n",
    "plt.legend(loc='upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wow！100%的准确率。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用核PCA进行非线性映射\n",
    "\n",
    "许多机器学习算法都有一个假设：输入数据要是线性可分的。感知机算法必须针对完全线性可分数据才能收敛。考虑到噪音，Adalien、逻辑斯蒂回归和SVM并不会要求数据完全线性可分。\n",
    "\n",
    "但是现实生活中有大量的非线性数据，此时用于降维的线性转换手段比如PCA和LDA效果就不会太好。这一节我们学习PCA的核化版本，核PCA。这里的\"核\"与核SVM相近。 运用核PCA，我们能将非线性可分的数据转换到新的、低维度的特征子空间，然后运用线性分类器解决。\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576d29033ad87.png)\n",
    "\n",
    "### 核函数和核技巧\n",
    "\n",
    "还记得在核SVM那里，我们讲过解决非线性问题的手段是将他们映射到新的高维特征空间，此时数据在高维空间线性可分。为了将数据$x\\in R^{d}$映射到高维k空间，我们定义了**非线性映射**函数$\\phi$:\n",
    "\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576d29bda8386.png)\n",
    "\n",
    "\n",
    "\n",
    "我们可以把核函数的功能理解为：通过创造出原始特征的一些非线性组合，然后将原来的d维度数据集映射到k维度特征空间，d<k。举个例子，特征向量$x\\in R^{d}$，x是列向量包含d个特征，d=2，可以按照如下的规则将其映射到3维度特征空间：\n",
    "\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576d2ea26cc6f.png)\n",
    "\n",
    "同理核PCA的工作机制：通过核PCA的非线性映射，将数据转换到一个高维度空间，然后在这个高维度空间运用标准PCA重新将数据映射到一个比原来还低的空间，最后就可以用线性分类器解决问题了。不过，这种方法涉及到两次映射转换，计算成本非常高，由此引出了核技巧(kernel trick)。\n",
    "\n",
    "使用核技巧，我们能在原始特征空间直接计算两个高维特征向量的相似性(不需要先特征映射，再计算相似性)。\n",
    "\n",
    "\n",
    "在介绍核技巧前，我们先回顾标准PCA的做法。我们按照如下公式计算两个特征k和j的协方差：\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576d30ef6b5c0.png)\n",
    "\n",
    "由于我们对数据已做过标准化处理，特征平均值为0，上式等价于：\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576d31fecd692.png)\n",
    "\n",
    "\n",
    "同样，我们能够得到协方差矩阵：\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576d323bf3249.png)\n",
    "\n",
    "\n",
    "Bernhard Scholkopf(B. Scholkopf, A.Smola, and K.R. Muller. *Kernel Principal Component Analysis*. pages 583-588, 1997)得到了上式的泛化形式,用非线性特征组合$\\phi$替换原始数据集两个样本之间的点乘：\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576d3314adfe3.png)\n",
    "\n",
    "\n",
    "为了从协方差矩阵中得到特征向量(主成分)，我们必须求解下面的等式：\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576d348dedb77.png)\n",
    "\n",
    "\n",
    "其中，$\\lambda, v$是协方差矩阵$\\Sigma$的特征值和特征向量，$ \\bf a$的求法见下面几段内容。\n",
    "\n",
    "\n",
    "我们求解核矩阵：\n",
    "\n",
    "首先，我们写出协方差矩阵的矩阵形式，$\\phi(X)$是一个n*k的矩阵：\n",
    "\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576d368a81005.png)\n",
    "\n",
    "我们将特征向量写作：\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576d36b56873b.png)\n",
    "\n",
    "由于$\\Sigma \\bf v=\\lambda \\bf v$，得：\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576d37678ff0a.png)\n",
    "\n",
    "\n",
    "\n",
    "等式两边左乘$\\phi(\\bf X)$：\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576d37a83bcaf.png)\n",
    "\n",
    "\n",
    "\n",
    "这里的$\\bf K$就是相似性(核)矩阵：\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576d37e0aca23.png)\n",
    "\n",
    "\n",
    "\n",
    "回忆核SVM我们使用核技巧避免了直接计算$\\phi(x^{(i)})^{T}\\phi(x^{(j)})$：\n",
    "\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576de7a56dc96.png)\n",
    "\n",
    "核PCA同样不需要像标准PCA那样构建转换矩阵，我们使用核函数代替计算$\\phi(\\bf X)\\phi(\\bf X)^T$。所以，你可以把核函数(简称，核)理解为计算两个向量点乘的函数，结果可看做两个向量的相似度。\n",
    "\n",
    "\n",
    "\n",
    "常用的核函数有：\n",
    "\n",
    "\n",
    "* 多项式核：\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576de8773bb6c.png)\n",
    "，$\\theta$是阈值，$p$是由用户设定的指数。\n",
    "\n",
    "\n",
    "\n",
    "* 双曲正切(sigmoid)核：\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576de8da01a63.png)\n",
    "\n",
    "* 径向基函数核(高斯核)：\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576de91e7e9f7.png)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "现在总结一下核PCA的步骤，以RBF核为例：\n",
    "\n",
    "1 计算核(相似)矩阵k，也就是计算任意两个训练样本：\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576de97a39675.png)\n",
    "\n",
    "得到K:\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576de9c97416f.png)\n",
    "\n",
    "举个例子，如训练集有100个样本，则对称核矩阵K的维度是100*100。\n",
    "\n",
    "2 对核矩阵K进行中心化处理：\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/24/576dea2e15413.png)\n",
    "\n",
    "其中,$1_{n}$是n*n的矩阵，n=训练集样本数，$1_{n}$中每个元素都等于$\\frac{1}{n}$.\n",
    "\n",
    "\n",
    "3 计算$\\bf K^{'}$的特征值，取最大的k个特征值对应的特征向量。不同于标准PCA，这里的特征向量并不是主成分轴。\n",
    "\n",
    "\n",
    "第2步为什么要计算$\\bf K^{'}$? 因为在PCA我们总是处理标准化的数据，也就是特征的平均值为0。当我们用非线性特征组合$\\phi$替代点乘时，我们并没有显示计算新的特征空间也就没有在新特征空间做标准化处理，我们不能保证新特征空间下的特征平均值为0，所以要对K做中心化。\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 用Python实现核PCA\n",
    "\n",
    "上一节我们讨论了核PCA的原理。现在我们根据上一节的三个步骤，自己实现一个核PCA。借助SciPy和NumPy，其实实现核PCA很简单："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy.spatial.distance import pdist, squareform\n",
    "from scipy import exp\n",
    "from scipy.linalg import eigh\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [],
   "source": [
    "def rbf_kernel_pca(X, gamma, n_components):\n",
    "    \"\"\"\n",
    "        RBF kernel PCA implementation.\n",
    "\n",
    "        X: shape[n_samples, n_features]\n",
    "        gamma: float. Tuning parameters of the RBF kernel\n",
    "        n_components: int. Number of pricipal components to return\n",
    "    \"\"\"\n",
    "    # Calculate pairwise squared Eculidean distance\n",
    "    sq_dists = pdist(X, 'sqeuclidean')\n",
    "    \n",
    "    # Convert pairwise distances into a square matrix\n",
    "    mat_sq_dists = squareform(sq_dists)\n",
    "    \n",
    "    # Compute the symmetric kernel matrix\n",
    "    K = exp(-gamma * mat_sq_dists)\n",
    "    \n",
    "    # Center the kernel matrix\n",
    "    N = K.shape[0]\n",
    "    one_n = np.ones((N, N)) / N\n",
    "    K = K - one_n.dot(K) - K.dot(one_n) + one_n.dot(K).dot(one_n)\n",
    "    \n",
    "    # Obtaining eigenpairs from the K'\n",
    "    eigvals, eigvecs = eigh(K)\n",
    "    \n",
    "    # Collect the top k eigenvectors(projected samples)\n",
    "    X_pc = np.column_stack((eigvecs[:, -i] for i in range(1, n_components+1)))\n",
    "    \n",
    "    return X_pc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "RBF核PCA的一个缺点是需要人工设置$\\gamma$值，调参不易。第六章我们会介绍调参技巧。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 例1 半月形数据分割\n",
    "\n",
    "现在我们创建一个非线性数据集试一下rbf_kernel_pca的效果，数据集100个样本，每个样本两维度特征，数据分布呈两个半月形："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import make_moons"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [],
   "source": [
    "X, y = make_moons(n_samples=100, random_state=123)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD8CAYAAACfF6SlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAHTxJREFUeJzt3X+wXOV93/H3B7Dw2IAsm2ugICxohQ1hNDW9g0lcO6LgqcwMUkoSF1qPoRZRXUonpFWmZGgoA/EEE5zSTAgpdT3GTmogrhwuRi4TgwiZjsG6pEIWMIBQbXEV2dyAIkSYQAjf/nF2rdXqnN29d8+en5/XjObuj0d7nrP7PN89+zzf8xxFBGZm1i5HlF0BMzMrnoO/mVkLOfibmbWQg7+ZWQs5+JuZtZCDv5lZCzn4m5m1kIO/mVkLOfibmbXQUWVXIMvxxx8fK1asKLsaZma18sQTT/xlREwNK1fZ4L9ixQpmZ2fLroaZWa1I+uEo5TzsY2bWQg7+ZmYt5OBvZtZCDv5mZi3k4G9m1kIO/mZmLZRL8Jf0ZUkvSdqR8bwk/Y6knZK2Szonj+2amdni5HXk/xVgzYDnPwms7PzbANyR03atyg4cgFtugddeG+1xMytMLsE/Ih4FXhlQZB3w1Ug8BrxH0kl5bNtKNiiQb9kCf/Znyd9RHh/2emaWm6LG/E8GXuy5P9d5zOoiKyhnBfIDB+CBB+CMM+Bb3zr4/7IeH+X1/KVglptKTfhK2iBpVtLs/Px82dVpp4UE+UGBfMsWePNNOPbY5G/3/2U9Psrr+UvBLDdFBf89wPKe+6d0HjtERNwZEdMRMT01NXRdIpuEhQT5rEDeLX/iicn9E09M/t/evemPj/p6C/lSMLOBigr+M8BnOlk/5wH7I2JvQdu2foMmYkcN8lkB/rXXDpY/+ujkuaOPTu7ffnv646O+3kK+FAbtp5nls6qnpK8Dq4HjJc0B/xl4B0BE/D6wGbgI2Am8DvyrPLZri9Q9Wj7zTLj44kMf7wbZV15J7q9enR6UX389O5A/+SREwA9+cOh2v/tdWL788Me3bUvKp73eAw/Aww8fvv3zz0+vb//+pO2nmeUT/CPisiHPB/Bv89iWjan/aPn88+GYY7KPvLOC/MwMvOMd6YH813994fW66ab0L4z77ku2uZAvhd796d9PMwMqvJ6/5eDAAbjjDrjqqoOBL+toOWuo5v774aijDg/KH/rQ4oJ8lqzXuukmeOGFw7c/MwNLlqT/8ujdn7RfBWnvi1nLOPg3Wf+wR9bR/fnnZw/VfPCD+Qb5hVrol8K2bdlDVd2jfw8HmTn4N1basEfW0f2WLeUG+MUYVN+Zmez97H4xeDjIWq5Sef62SGlZLWkZMr1H991/EcnRcpMM2s9h5xk4O8hawkf+TTDq8M4Xv9iOo9ysXwUHDsDGjR4OMsNH/vWXlus+aHinzQa9L8POGTBrGB/5111aVkvW5O22be0+oh30vnTPM8g6Z8CsYRz866Y3TTGi3cM7C7XY4SCnhloDedinbnrXsvHwTj6GvY9eP8gayMG/TvrHpb/3vXZk70zaoOwgzwVYQ3nYp076x/c/8hH4jd8ou1b1N8o5A54LsIbxkX9V9eecD1r10iZj0HvucwKs5hz8q6p/nNnj+8Ub9J57HsBqzsG/itLGmdtydm6VZL3njz/ueQCrPY/5V1Fa7n7d1t5pgqz3fGYGvvlNzwNYrfnIvwp6x489tl9twz4fzwVYTTj4V4Fz9+vD5wRYQ+QS/CWtkfSspJ2Srk15/lRJWyT9X0nbJV2Ux3Ybwbn79eJzAqwhxh7zl3QkcDvwCWAO2CppJiKe7in2n4B7I+IOSWeRXNN3xbjbbgTn7teLzwmwhsjjyP9cYGdE7IqIN4G7gXV9ZQI4rnN7KfAXOWy3/jy+3xz+LK1m8gj+JwMv9tyf6zzW6wbg05LmSI76/10O262f/slAj+83x7Dloj0JbBVT1ITvZcBXIuIU4CLga5IO27akDZJmJc3Oz88XVLUC9U8GOne/OYZdPcyTwFYxiojxXkD6aeCGiPinnfu/BhARv9lT5ilgTUS82Lm/CzgvIl7Ket3p6emYnZ0dq26V0l02+JhjkiNAL7vcDv7crWCSnoiI6WHl8jjy3wqslHSapCXApcBMX5ndwAWdip0JvBNo4KH9AIOuHWvN5c/dKmrs4B8RbwFXAw8Cz5Bk9Twl6UZJazvF/gPwS5KeBL4OXBHj/uSoE08GtpM/d6uwXJZ3iIjNJBO5vY9d33P7aeCjeWyrlgZNBjoVsLn8uVuF+QzfSenN8PDEbjsNOyHMGUBWIi/sNindDI8zz/SibG017ISwbvvwrwArgY/8J8Gn+dsgbh9WAQ7+k+AMDxvE7cMqwME/b87wsEHcPqwiHPzz0p3A27zZSzZYtqwMoAce8ASwFcoTvnnpTvBu3w5HHZVkdvTats0Te3ZoBlCvmRl49VVPAFthHPzz0DuB51P4bZC0DKDuEhDdIaDzz3f7sYnzsE8ePIFn43D7sRI4+I/LE3g2DrcfK4mD/7i8Jr+Nw+3HSuLgv1jd7J6tW710gy2el4CwknjCd7G62T0bNsBNN5VdG6srLwFhJfGR/2L49HybNLcxmzAH/8VwdoZNmtuYTZiD/0I5O8MmzW3MCuDgv1DOzrBJcxuzAuQS/CWtkfSspJ2Srs0o8ylJT0t6StL/zGO7pfCFWWzS3MasAGNn+0g6Ergd+AQwB2yVNNO5dGO3zErg14CPRsQ+Se8fd7uFO3AA7rgDfuVXfOq9TVZWBlA39fOqq9wGbWx5HPmfC+yMiF0R8SZwN7Cur8wvAbdHxD6AiHgph+0Wq5va6Z/eVha3QctRHsH/ZODFnvtzncd6nQGcIen/SHpM0poctlscp91Z2dwGLWdFTfgeBawEVgOXAf9d0nv6C0naIGlW0uz8/HxBVRuB0+6sbG6DlrM8gv8eYHnP/VM6j/WaA2Yi4m8j4v8Bz5F8GRwiIu6MiOmImJ6amsqhajlw2p2VzW3QJiCP4L8VWCnpNElLgEuBmb4yf0xy1I+k40mGgXblsO3Jc9qdlc1t0CZg7OAfEW8BVwMPAs8A90bEU5JulLS2U+xB4GVJTwNbgF+NiJfH3XYhnHZnZXMbtAlQRJRdh1TT09MxOztbXgW6qZ1Oq7Mqcvu0DJKeiIjpYeV8hm8Wp9VZlbl92pgc/NM4rc6qzO3TcuDgn8ZpdVZlbp+WAwf/fk6rsypz+7ScOPj3c1qdVZnbp+XEl3Hs15tW12vbNl9Kb5G2b4dNm2D3bjj1VLjkEli1quxa1ZTbZyOV0Uec6tnl1LmJ2L4dbr0Vli2DpUth/37Ytw82bvQXQK7cfmsr7z7iVM+FcurcRGzalDTqZcvgiCMO3t60qeyaNYzbb22V1Ucc/MGpcxO0e3dyNNNr6dLkccuJ22+tldVHHPzBqXMTdOqpyc/YXvv3J49bTtx+a62sPuLg79S5ibrkkmT8ct8+ePvtg7cvuaTsmjWE22/tldVHHPydOjdRq1YlE1fLlsHcXPLXk705cvutvbL6iFM9nTo3catWOdhPjNtvI5TRRxz8r7nGKXIT5jz/CUq72Hs37fO119yma6KMPuJhH6fITVQ3h3nfPjjllOTvrbcmj9uEuE3XSll9pN3B3ylyE+c8/4K5TdeO8/zL4BS5iXOef8Hcpmun1nn+ktZIelbSTknXDij385JC0tBTjyfOKXKFcJ5/gdyma6m2ef6SjgRuBz4JnAVcJumslHLHAr8MPD7uNnPhFLlCOM+/QG7TtVTnPP9zgZ0RsSsi3gTuBtallLsJ+ALwNzlsc3y+KHYhnOdfILfpWqpznv/JwIs99+eAj/QWkHQOsDwiHpD0qzlsc3xpKXI2Ec7zL4jbdG01Ms9f0hHAbwNXjFB2A7AB4NRJDnh5+dvCOde/BG7ntVBW38hj2GcPsLzn/imdx7qOBc4GHpH0A+A8YCZt0jci7oyI6YiYnpqayqFqGZwHXSjn+pfE7bzyyuwbeQT/rcBKSadJWgJcCsx0n4yI/RFxfESsiIgVwGPA2ogo8EotPZwHXTjn+pfA7bwWyuwbYwf/iHgLuBp4EHgGuDcinpJ0o6S1475+7pwHXTjn+pfA7bwWyuwbueT5R8TmiDgjIv5+RHy+89j1ETGTUnZ16Uf9zoMulHP9C+Z2Xhtl9o12neHrPOhSONe/YG7ntVFm32hX8HcedCmc618wt/PaKLNvKCImv5VFmJ6ejtnZckaHzMzqStITETF0CZ12refvvOdSOde/JG73lVVmn2jXsI/znkvjXP8Sud1XUtl9oj3B33nPpXKuf0nc7iur7D7RnuDvvOdSOde/JG73lVV2n2hH8Hfec+mc618Ct/tKK7tPtCP4O++5dM71L4HbfaWV3SfaEfyd91w65/qXwO2+0sruE+3I83eqm5n7QUs4z79XN9XtzDPh4ovLrk2rOde/RO4HlVJ2X2j+sI9T3Sqj7LzmVnM/qJQq9IXmB3+nulVG2XnNreZ+UClV6AvNDv5OdauUsvOaW8v9oHKq0BeaHfyd6lYpZec1t5b7QeVUoS80O/g71a1Sys5rbi33g8qpQl/IJdVT0hrgvwJHAl+KiJv7nv/3wJXAW8A88NmI+OGg1/SSzs1UdoaDWVVMqi+Mmuo5dvCXdCTwHPAJYI7kgu6XRcTTPWXOBx6PiNcl/RtgdUT880Gv6+BvZrZwReb5nwvsjIhdnQ3fDawDfhL8I6J3cPEx4NM5bHc4n9RSWf4FUCL3i1JVpe3nMeZ/MvBiz/25zmNZ1gPfTntC0gZJs5Jm5+fnx6+Z1zGvpCrkOLea+0VpqtT2C53wlfRpYBr4rbTnI+LOiJiOiOmpqanxNuaTWiqrCjnOreV+Uaoqtf08gv8eYHnP/VM6jx1C0oXAdcDaiHgjh+0O5pNaKqsKOc6t5X5Rqiq1/TyC/1ZgpaTTJC0BLgVmegtI+jDw30gC/0s5bHMwn9RSaVXIcW4l94vSVantjx38I+It4GrgQeAZ4N6IeErSjZLWdor9FnAM8EeStkmayXi5fPiklkqrQo5zK7lflK5KbT+XVT0jYjOwue+x63tuX5jHdkbWe1JLr23bvJphBXTXMe/NeFi/3tk+E+d+Uboqtf12rOdvZtYSXs/fucy1UJWc59Zx/yhc1dp6c9f2cS5z5VUp57l13D8KVcW23szg71zmWqhSznOruH8UroptvZnB37nMtVClnOdWcf8oXBXbevOCv3OZa6NKOc+t4f5Riiq29eYFf+cy10aVcp5bw/2jFFVs680L/r5wRW10c56XLYO5ueTvxo3O9pko949SVLGtO8/fzKxBnOfvPObaqVoedKO5fxSiym26ecM+Xc5jrpUq5kE3mvvHxFW9TTcz+DuPuXaqmAfdWO4fhah6m25m8Hcec+1UMQ+6sdw/ClH1Nt284O885lqqYh50I7l/FKbqbbp5wd95zLVUxTzoRnL/KEzV23Tzgr/zmGupinnQjeT+UZiqt2nn+VslVTlFzmyQstvuqHn+uRz5S1oj6VlJOyVdm/L80ZLu6Tz/uKQVeWx3oAMH4JZbPJZZQ1VPkWsM95Hc1antjh38JR0J3A58EjgLuEzSWX3F1gP7IuIfAP8F+MK42x3Kecy1VfUUucZwH8ldndpuHkf+5wI7I2JXRLwJ3A2s6yuzDrirc/sbwAWSlMO20zmPudaqniLXCO4jE1GntptH8D8ZeLHn/lznsdQyEfEWsB94X/8LSdogaVbS7Pz8/OJr5DzmWqt6ilwjuI9MRJ3abqWyfSLizoiYjojpqampxb2I85hrr+opcrXnPjIxdWq7eQT/PcDynvundB5LLSPpKGAp8HIO2z6c85hrr+opcrXnPjIxdWq7eazquRVYKek0kiB/KfAv+srMAJcD3wV+AXg4JpVj2pvH3GvbNrj44ols0vK3atXBDtNNnbvtNqd95sJ9JHdlp3cuRi55/pIuAm4DjgS+HBGfl3QjMBsRM5LeCXwN+DDwCnBpROwa9JrO8zc4mDq3bFkycbZ/f/IzuqpHU9Y+VWujha7nHxGbgc19j13fc/tvgF/MY1vWLr2pc3Dw76ZNDv5WDXVto5Wa8DXrV6fUOWunurZRB3+rtDqlzlk71bWNOvhbpdUpdc7aqa5t1MHfKq1OqXPWTnVto17V02qnjml11ixVboOFruppVpQ6rZpozdSUNujgb7VSp1UTrZma0gYd/K1W6ppWZ83RlDbo4G+1Ute0OmuOprRBB3+rlbqm1VlzNKUNOtvHaqc/0+Lss2HHjmpmXlgz1KnNjZrt4+BvtVa1RbWseerWxpzqaa3QlMwLq66mtjEHf6u1pmReWHU1tY05+FutNSXzwqqrqW3Mwd9qrSmZF1ZdTW1jDv5Wa3VdVMvqo6ltbKxsH0nvBe4BVgA/AD4VEfv6yvxD4A7gOODvgM9HxD3DXtvZPrZYVV50y+qhzm2oqGyfa4GHImIl8FDnfr/Xgc9ExE8Ba4DbJL1nzO2apWrKoltWnra0oXGD/zrgrs7tu4Cf6y8QEc9FxPOd238BvARMjblds1RNTcuz4rSlDY0b/E+IiL2d2z8CThhUWNK5wBLghYznN0ialTQ7Pz8/ZtWsjZqalmfFaUsbOmpYAUnfAU5Meeq63jsREZIyJxAknQR8Dbg8It5OKxMRdwJ3QjLmP6xuZv1OPTX5mb5s2cHHmpCWZ8VpSxsaGvwj4sKs5yT9WNJJEbG3E9xfyih3HPAAcF1EPLbo2poNccklyfgsHHoq/sc+BjfcUM8JPJustHV7ZmaS53rb0Pr15dYzb+MO+8wAl3duXw7c119A0hLgm8BXI+IbY27PbKC0tLy1a5PO3PQJPFu4tMndmZmkzTQttbPf0CP/IW4G7pW0Hvgh8CkASdPA5yLiys5jHwfeJ+mKzv+7IiK2jblts1SrVh3aUW+44eCkHRz8u2lT8zq0LUzv5C4c/LtjR9Jummys4B8RLwMXpDw+C1zZuf0HwB+Msx2zcezenRzV9WriBJ4tXJvbhs/wtcZr6tosNr42tw0Hf2u8pq7NYuNrc9vwxVysFdJO14f6nsJvi9OGduAreZkNULerM9n42vKZ+0peZgO05RR+O8if+aEc/K2V2nIKvx3kz/xQDv7WSm3O8mgrf+aHGvckL7Na8jIQzdfWZRtG5SN/ayUvA9FsbV62YVQ+8rfW8jIQzdXmZRtG5SN/sw5PCDaHP8vhfORv1pG1jvuSJZ4HqLr+8f0lS5LPrulr8o/DR/5mHWmn+r/wAuzZ43mAKksb39+zJ/ns2rhsw6gc/M060iaBly+H00/3iUFVlnby1umnJ5+dJ3ezedjHrEf/JPBnPwtTU4eW8dhxtWQtyzw358ndQRz8zQYYdD3XtEXCfGQ5WWnveVuuuZu3sYZ9JL1X0p9Ier7zd9mAssdJmpP0u+Ns06xIWUv+nn324ePMnguYrLSx/VtvTT6Lti7LPI5xx/yvBR6KiJXAQ537WW4CHh1ze2aFSpsH2LgxyRf3ImHFylqYbceO9M/Iv8IGG3fYZx2wunP7LuAR4D/2F5L0j4ATgP8NDF1q1KxK+ucBAG67rb2X/yvLoEsupn1GNti4wf+EiNjbuf0jkgB/CElHAF8EPg1cOOb2zCrBcwGT5bH9yRs67CPpO5J2pPxb11sukqvCpF0Z5ipgc0TMjbCtDZJmJc3Oz8+PvBNmRfNcwOR4bL8YY13JS9KzwOqI2CvpJOCRiPhgX5k/BD4GvA0cAywBfi8iBs0P+EpeVnlpR6ebNh1+dNq977TD0dxwQ/Z72H2P/asq26hX8hp32GcGuBy4ufP3vv4CEfEveyp1BTA9LPCb1cFC5wI8HHS4tPfEY/vFGDfb52bgE5KeJxnPvxlA0rSkL41bObO6ybpgyNFHezioX9bwTnddnl4e28/fWME/Il6OiAsiYmVEXBgRr3Qen42IK1PKfyUirh5nm2ZVljUXEOHU0H5ZqZuSx/aL4DN8zXLUPS+gdyhj/fpkOOj97z+0bJuGgxYyvDM3l/4eNu09KdtYE76T5Alfa5KsScw334S//uvk8d5LCzbpJKXu8E7/Pr773ckQjyfH81XUhK+ZjSDrmsHvelf21cO6f+v0iyArAyptH994I3kPwNfULYOP/M0KkhYYu9lBR/TMvr39dlL2uOOyfxGUPVSUtn1IP8J/9dWkbv37ODcH11xTvy+4qvORv1nFpKUpZp21+ld/BR/4QPYvgm6Q7c2S2bjxYJk8gmnWF0zvME7v9t/97vQj/N27s6+q5dTN8vjI36xEWePhg46W074w9u1LhlFefz391wJkfyks5Ci+OxGbtv1HH4WLL174rxjL16hH/g7+ZiVb6JnC3SyZ/iA7MwM/+7MLm1SG9CD/rncl5ybksX2fmVssD/uY1UTW0EfaBPH69elfDPv3J/nxS5ce+hpLl8L998PHP549hJQ2VPOnfwpr1x7+Wt3gnbb9887LnsD18E71+Bq+ZhWUdR2BVauyTyQ777z0M2Mj0r8Udu9O/qU9J2WfZZu1/auu8rr6deJhH7MaWsg4/aAhnP7b3fvDzj8oO9vIsnnM36yFFjp5O+w5B/j6cfA3s58YdKTuo/hm8YSvmf3EoAlXT8a2kyd8zcxayMHfzKyFHPzNzFrIwd/MrIUc/M3MWsjB38yshRz8zcxaqLIneUmaB3445sscD/xlDtUpm/ejWpqyH9CcffF+HPSBiJgaVqiywT8PkmZHOdOt6rwf1dKU/YDm7Iv3Y+E87GNm1kIO/mZmLdT04H9n2RXIifejWpqyH9CcffF+LFCjx/zNzCxd04/8zcwsRaOCv6RflPSUpLclZc6YS1oj6VlJOyVdW2QdRyHpvZL+RNLznb/LMsr9naRtnX8zRdczy7D3V9LRku7pPP+4pBXF13K4EfbjCknzPZ/BlWXUcxhJX5b0kqQdGc9L0u909nO7pHOKruMoRtiP1ZL293we1xddx1FIWi5pi6SnO/Hql1PKTP4ziYjG/APOBD4IPAJMZ5Q5EngBOB1YAjwJnFV23fvqeAtwbef2tcAXMsq9VnZdF/P+AlcBv9+5fSlwT9n1XuR+XAH8btl1HWFfPg6cA+zIeP4i4NuAgPOAx8uu8yL3YzXwrbLrOcJ+nASc07l9LPBcStua+GfSqCP/iHgmIp4dUuxcYGdE7IqIN4G7gXWTr92CrAPu6ty+C/i5EuuyUKO8v7379w3gAkkqsI6jqEM7GUlEPAq8MqDIOuCrkXgMeI+kk4qp3ehG2I9aiIi9EfHnndsHgGeAk/uKTfwzaVTwH9HJwIs99+c4/I0v2wkRsbdz+0fACRnl3ilpVtJjkqryBTHK+/uTMhHxFrAfeF8htRvdqO3k5zs/y78haXkxVctdHfrEqH5a0pOSvi3pp8quzDCdIc8PA4/3PTXxz6R2l3GU9B3gxJSnrouI+4quz2IN2o/eOxERkrJSsj4QEXsknQ48LOn7EfFC3nW1TPcDX4+INyT9a5JfM/+k5Dq12Z+T9InXJF0E/DGwsuQ6ZZJ0DPC/gGsi4tWit1+74B8RF475EnuA3iO0UzqPFWrQfkj6saSTImJv56feSxmvsafzd5ekR0iOIMoO/qO8v90yc5KOApYCLxdTvZEN3Y+I6K3zl0jmauqoEn1iXL0BNCI2S/o9ScdHROXW/JH0DpLA/4cRsSmlyMQ/kzYO+2wFVko6TdISkgnHymTKdMwAl3duXw4c9otG0jJJR3duHw98FHi6sBpmG+X97d2/XwAejs4sV4UM3Y++Mdi1JGO3dTQDfKaTYXIesL9n2LE2JJ3YnTuSdC5JfKvaQQWdOv4P4JmI+O2MYpP/TMqe+c55Fv2fkYyNvQH8GHiw8/jfAzb3zaQ/R3KUfF3Z9U7Zj/cBDwHPA98B3tt5fBr4Uuf2zwDfJ8lC+T6wvux6D3p/gRuBtZ3b7wT+CNgJfA84vew6L3I/fhN4qvMZbAE+VHadM/bj68Be4G87/WM98Dngc53nBdze2c/vk5EpV/a/Efbj6p7P4zHgZ8quc8Z+/GMggO3Ats6/i4r+THyGr5lZC7Vx2MfMrPUc/M3MWsjB38yshRz8zcxayMHfzKyFHPzNzFrIwd/MrIUc/M3MWuj/A0Yxxgz7DeQXAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff3a5134390>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(X[y==0, 0], X[y==0, 1],\n",
    "           color='red', marker='^', alpha=.5)\n",
    "\n",
    "plt.scatter(X[y==1, 0], X[y==1, 1],\n",
    "           color='blue', marker='o', alpha=.5)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "每个半月数据是一类。\n",
    "\n",
    "显然图中的数据不是线性可分的，我们的目标是通过核PCA将\"曲线\"映射到\"直线\"上，然后就可以用线性分类器了。\n",
    "\n",
    "我们先看一下如果用标准PCA处理，会得到什么结果："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.decomposition import PCA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [],
   "source": [
    "scikit_pca = PCA(n_components=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_spca = scikit_pca.fit_transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcUAAADUCAYAAADtPDgOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3X90VOd54PHvI4GEMQiEkQGDBa4DDjaRMVExbhqME5M4SW02tOsfbX41Tjmpj5v+2Oyps6mznLjpetNkT7anjrdO4rpk07huQmzZJvHGLmC3sTHCBgIGLGKDLEBCgJDED0lIevaPZ27mMox+wYzu3LnP5xydub80885I9z7zvu/zvldUFeecc85BSdQFcM455wqFB0XnnHMuxYOic845l+JB0TnnnEvxoOicc86leFB0zjnnUiINiiJyi4jsEZG9InJflv3VIrJeRF4Xke0i8tEoyumccy4ZJKpxiiJSCrwJLAeagM3AXar6RuiYR4DXVfVhEbkaWKeqc6Ior3POueIXZU1xMbBXVd9S1R7gcWBFxjEKVKSWJwEHR7F8zjnnEibKoDgTeCe03pTaFrYa+ISINAHrgD/J9kQiskpE6lM/q/JRWOecc8VvTNQFGMJdwGOq+k0RuQH4vogsUNX+8EGq+gjwCMDUqVO1trb2HyIoq3M5s2XLliOqWhV1OQJTp07VOXPmRF0M587bcM+pKIPiAeDy0Pqs1Lawu4FbAFT1ZREZB0wFDg/0pHPmzKG+vj7HRXVudInI/qjLEObnlYu74Z5TUTafbgbmisgVIlIG3AnUZRzTCHwQQETmA+OA1lEtpXPOucSILCiqai9wL/AcsAt4QlV3ishXReS21GH/BfgjEdkG/BD4jPptPZxzzuVJpH2KqroOS6AJb/tKaPkN4H2jXS7nnHPJ5DPaOOeccykeFJ1zzrmUQh+SkVfbt8PatdDYCNXVsHIl1NREXSrnnHNRSWxNcft2+MY3oK0NZs2yx298w7Y755xLpsQGxbVrobLSfkpK0str10ZdMuecc1FJbFBsbIRJk87eNmmSbXfOOZdMiQ2K1dXQ3n72tvZ22+6ccy6ZEhsUV660fsS2NujvTy+vXBl1yZxzzkUl0dmn48fDxo0gAkuWwBe/6NmnzjmXZIkMikHmaWUl3HabNZu2tUVdKuecc1FLZPOpZ54655zLJpFB0TNPC0xnJ3z963DiRPZ155wbJZEGRRG5RUT2iMheEblvgGNuF5E3RGSniPxzLl7XM08jlhn01q+Hl16yx8x1D5jOuVEUWVAUkVLgIeAjwNXAXSJydcYxc4EvAe9T1WuAP8vFa3vmaQTCwSwz6D37LMybB888A4cOnb2+bt3AAdM553IsypriYmCvqr6lqj3A48CKjGP+CHhIVdsAVPVwLl64psYyTSsroanJHj3zNMcGqg0+++y5Qa+nByZOtMeHHkqvnzwJjz46cMA8dMhrjc65nIoyKM4E3gmtN6W2hc0D5onIf4jIKyJyS7YnEpFVIlIvIvWtra15Kq4bkYFqg48+asEuHPSmT7ffqayEH/8Ypkyx9a4uePttKC8/N2AG6+FaozetOucuUKEn2owB5gLLgLuA74jI5MyDVPURVa1V1dqqqqohn9QnA8+TIChlawLt6YGyMti3z4IdpIOeiK0fOgRnzsDBg9DdbfvGjbPHzIAZrM+ZY6+R2SzrnHPnIcqgeAC4PLQ+K7UtrAmoU9Uzqvo28CYWJC+ID8nIkyAoZTaB/uM/Wm1w3z4LjG+/bfuPHrVO3U2bbN/u3fY8u3fbtvZ2+wO1tJwdMCG9fuSIPVdms6w3rTrnzkOUQXEzMFdErhCRMuBOoC7jmCexWiIiMhVrTn3rQl/Yh2TkSLi5MmginTMnexOoCDQ3Q2mpBbtNm2D2bPjt34Zly+Cxx+D112HvXnu86Sbbt3ChpQWHA2ZDA2zZAhddZAFz+vSzm2WzNa0659wwRDajjar2isi9wHNAKfCoqu4Uka8C9apal9r3IRF5A+gD/quqHr3Q166utibTysr0Nh+ScR6CmuH8+aBqwaijI12ju+oqOHYM+vrSQXD2bPvdK6+E++8f+LkH21dXZzXO4LmCptaJE209aFpdvtxqjTfdBBMm5OY9O+eKWqTTvKnqOmBdxravhJYV+IvUT86sXGl9iGA1xGCat7vvzuWrFKnOTnj4YfjkJ9PNlWvXWk1w+nR45RWrwW3ZYk2f1dX2M1QQHIlt2ywI79tn6++8Y82wR1Pfl4Km1dZW65Ncv95qow8/DPfc4wHSOTegRM59CnDxxfDii3Zt9cnARyCoHXZ0pPsNt261oFhdDTfeaMft32/fPm69NfdlyAyuDzwAv/qVLYebVpub7Y/7zDNw6lS6VpuPMjnnikLigmJ4MvBbb/XJwEcks9/wox+17WfOWABqaICxY9PHb906OgEoHCQzm1YhPfRjwQILkLW18P3ve63ROXeOxAXFcOYppB/XrvWa4oCCJtPZs7P3G37gA/mtGY5EZtMqWDnb2qxWe+yYJeFs2+a1RufcORIXFBsbbWximGeeDmH9evtRheuuO7ffMKgdjlbNcDCZTaudndY2njm+0ZNwnHNZJC4oeubpCAVNpmVlNhxi0aLR6TfMlfXrrXZbXm7r2ZJwCrn8zrlRVegz2uScTwY+TMEYxGA2ms7O9NCKffvsR9Vqh4Us3JyamYQzfbq1mz/wgA/yd84BCawpBpOBr11rTabV1TYUw/sTMwRNpv39VjucPdvGAx47Bt/8ZnyaHIdKwmlshB07bJIArzE6l3iJqym6YQiaTMvLz56bNJiYO66zxIRrjUHN8Z13LFAG86c65xItcTXF8JCM8GTgPk4xJOiHa29Pz016eWia2kJIqDkfmUk44Zrj/v32RWD/fh+q4VyCJS4o+pCMQXR2wre+ZbXD6dPj22Q6HEFtOLhtVTB/akmJD9VwLsES13zqk4EPYv16qz3t35/O1ox7k+lAMrNSIX3vRm9KdS6xIg2KInKLiOwRkb0ict8gx/2uiKiI1F7oa1ZXW6tgmA/JIF1zGjvW+tkaGuKVZTpSmf2Lr75qTcXt7cX5JcA5NyyRNZ+KSCnwELAcu2/iZhGpU9U3Mo6bCPwpsCkXr+uTgQ8gqDl96EPxGH94ocL9i8EA/8WLrabY3W3t6du2wZ/9WXE1GzvnBhVlTXExsFdV31LVHuBxYEWW4x4A/ifQlYsXramB226z690Pf2iPt92W4P7Ezk4bp/fkk2f3ryWpCTGzKbW83NrTn3rKa4zOJUyUQXEm8E5ovSm17ddEZBFwuao+m6sX3b7dus2uvRbuusse6+pseyIF/YiNjcXfjziQgYZqjB2brC8HzrnCzT4VkRLgfwGfGcaxq4BVANVDdA569mlIeAq3xsbo7nIRtaGGavhUcM4lRpRB8QAQGvzGrNS2wERgAbBBbPD4dKBORG5T1frwE6nqI8AjALW1tTrYi/qE4CFBs+Hy5cnoRxyObEM11q61Lwh//ufev+hckYuy+XQzMFdErhCRMuBOoC7YqartqjpVVeeo6hzgFeCcgDhSnn2K9yMOZqD+xbq65DQnO5dgkQVFVe0F7gWeA3YBT6jqThH5qojclq/X9QnB8X7EwfhUcM4lWqR9iqq6DliXse0rAxy7LBevmfgJwb0fcXDev+hcohVsoo3LE+9HHL5s/Yt+Y2LnilripnkLJgRvazt7QvBEDMkY6CLvTYLZZetfPHkS7r3XPzPnilTigmJ4SEZJSXp57dqoS5ZnnZ3wJ39iF3XvRxyezP7Fffvg0CF4+WX/zJwrUolrPk3skIz16+1iPmsWjMn4sye9H3Egmf2LwXRwCxd6M6pzRSpxQbG62ppMg0H7kIAhGUGz6e/8jjX7FdttoEZL0Jw6caLdTsuTbpwrOolrPk3kkIzwxdybS8+P98c6lwiJC4rBkIzKSmhqsscvfrGIh2T4xTw3st1/ccsWWLdu4N9xzsVO4oJi4qxbZxdvmyrPk2vOV2bSzaZN0Nxs4xidc0UjcX2KwZCMysqzh2QUbW2xrs4u3ps2weWhqWY9uWZkst1/8brrrMZ94oT30TpXJBIXFBN1l4zOTrtY33GHJ9jkkifcOFe0Etd82thoQzDCinZIhifY5J730TpX1BIXFBNzlwy/eOeHJ9w4V9QiDYoicouI7BGRvSJyX5b9fyEib4jIdhF5QURmX+hrJmZIhifY5Edmws2rr0JLCzz1VNQlc87lQGR9iiJSCjwELAeagM0iUqeqb4QOex2oVdVTIvLHwNeBOy7kdYMhGQ8/DE8/bde3JUsu5BkLlCfY5Ee2hJuFCz3hxrkiEWWizWJgr6q+BSAijwMrgF8HRVUNV2teAT6Rqxc/eRKWLrX+xPb2IstA9QSb0eEJN84VnSibT2cC74TWm1LbBnI38NNsO0RklYjUi0h9a2vrkC9c9JOCe4JN/nmfrXNFKRaJNiLyCaAW+Nts+1X1EVWtVdXaqqqqIZ+vqDNQ/WI9Ovy2Us4VpSiD4gEg1NnFrNS2s4jIzcCXgdtUtTsXL1zUGajZLtZeW8y9bLeVOnjQbyvlXMxF2ae4GZgrIldgwfBO4PfDB4jIdcA/ALeo6uFcvfDKldaHCOk+xbY2uPvuXL1ChF59FX75SwuEY8emt3uCTW4NdFup667z20o5F2ORBUVV7RWRe4HngFLgUVXdKSJfBepVtQ5rLp0A/KvY0IJGVb3tQl87yED99rctSVOkiDJQFy+2Wsydd3oQHE2edONcUYh0mjdVXQesy9j2ldDyzfl8/VOn4MYbiygDNehPnDfPayujaaB+XP/8nYudWCTa5ENRZqB61mk0vB/XuaKR2KBYdBmonnUanXDSTUODfe49PdaP65yLlcTdJSNQXW3JNcFdMiDmGajPPmvTun3oQ7Yerq1431Z+hZNu6urgO9/xPl3nYiqxNcWimwO1rs7m4Ny0KT1EQLWoaivbt8Pq1fDZz9rj9u1RlyhDZp+u19Kdi53E1hSDDNS1a63JtLrahmTEMsmms9P6EW+/vWindYvFzaE9A9W52EtsTRHsYrpypQXExkYLkAVX+xiOBCTYFHxilPfpOlcUEh0Ug9pHW9vZtY9YBcaEXIwLPjHKM1CdKwqJDooFX/sYjoRcjAt+ar7Mad8aGuzb1auvRl0y59wIDNmnKCIVQJWq/ipje42qxqlOdY7GRqshhhVU7WM4EjKtW8FPzZc57VuQhbp4cTTlcc6dl0GDoojcDnwLOCwiY4HPqOrm1O7HgEX5LV5+FcWwjARN63bxxfDii+kbQxdUkk2YzyzkXGwN1Xz634D3qupC4A+B74vIx1P7JK8lGwWxH5aRkCEAQd9vWZnF/RtvtCn6ClYCEp+KUmcnPPCA1fr/+q/h0CH4+teHfjxxwn4318tBmYazbyTH5uN5crE80Of6wAPn/j3+6q/sQrBkiTX3TZmSs2SQoZpPS1X1EICqvioiNwHPiMjlgF7oi4vILcD/xiYE/66qPpixvxxYA7wXOArcoar7LvR1A8GwjIcfhqefTtdAYiMhQwDCfb+Qfly7tgBrij4PanytX2/N3kHzUVeXtcJ0dAz+OH++XTxeeim3y7feamUazr6RHJuP58nF8kCfa12d/X3Cf4/HH4emJrv+aSoU3XMP/Pu/X/C/gagOHNtE5BfAJ8P9iSIyEXgS+G1VLT/vFxYpBd4ElgNN2K2k7lLVN0LH3APUqOrnReRO4OOqesdgz1tbW6v19fXDLkd4/Fu4r6pgm+YCwa2Kpkyx5JrubguMRThG8bOftS+DJaF2jf5+OycefTS6cmVVVwc/+QnMnp3etn+/NT+M4AuLiGxR1do8lPC8jPS8ip3OTvjCF6yP/tAhmDbN/smWLbM2+6VLz3688UbYuBGWL7ffFbGLyLFjuVk+ccJmqFi92s7nwfaN5Nh8PE8uli++GJ5/Pv35Bp9rb68Fwr4+W162DF54AVpbLTiGlZbCa68NeOEe7jk1VE3xj8loJlXVzlQN7/ahnnwIi4G9qvoWgIg8DqwA3ggdswJYnVr+EfD3IiI6WCQfoVjVQsIGyzotstpirPp+w1moAGfOwO7dFtWL7O8S9qlPwQ9+YHEkMGYMjB8PkydDVZU1aKjaNTD8OHkyzJhhyz091kwuYrHp+PGB93d32//AggWwY4clyJWXD++4c/ad2sWODctoPPJRyro7kNMldMs4qn92mgVdpez4WS2Nbcup/lmHrf+0lsa2myl7tgLpOk13fxnVV41jQesGdpz+DRon1VDW0oiI0H3p5VQf386C8W+xo2oZjXu6KJMe5LIZdB84SvVFh1nw3nJ21HfT2HUpZTMvQTo76b7lIGWn70AqJtB94CjlJT3ojBn0tHdTtnk/cvJ2usdPoexkG7K5kUOtqzheOoXJJw8wo+woOmMGzS2lHP/ZaaT7T9FxFyHH2+jsHc+J8iloby9jfiz09vx3LhovLKnYxT39P6KmN9X6tHWrfUjV1Ra8Hnoo3TIVzJRVXX32cYNtH+yYqio7V3btssfWVjh82B5LSy1Adnfb/iNH4PTpc/8J+/pyUlscKiieBKYBezO2LwZeuaBXhpnAO6H1JuD6gY5J3X+xHbgEOHKBr/1rsc1A3bbN/kGffhre/e505mmRZZ1CDDJPwxKYhfqpT8H3v3/u9t5e+zJ/8qRdx0pKrGGjo8Ouq52dcOmlcOCA/TuPGwfXXAM7d1pL2dixFlSz7QerVDQ0wJo11u0xcSJs2GD7Mo9780077oYbrKKzcWPoOd7oYc1PZ7OkpIWJZ46xsf/9cKaPpWNfoeHQBNbIl1jS/wrvuuQYbx6sZE3Jl7hBX2ZChbDxwLtAYenFW2hoqWLNrz7HkvHbmXjxKTYee4/tu+QQDW1TWXPgAyzpbWFi9xE2nnovdI1nKftpOD2VNf/vepbIJiaWnGDj24tAlWu6X2dn+XVwtIQFZ9p4ufs90HUx11QeZOeOiTBhDtdMO8LOI1fT9dYZxk4sY/zYXg60z2KbzoGTFzFGeyk5dYIWmcPkcac5dmIqqkrJmTH09Qs9vSWML+tjZnkHG49cTdM/w9/8bhc13cfg6FH7kHp67Bvpj38MH/2oBadgX2fnhS/39dnjlCnwxhtwxRXp4NjcDFdeaUGwt9cy7U+ftn3ZvPKKNf9dQI1mqKD4LeBLWbZ3pPYVxNVXRFYBqwCqR1h9iFUtJOz++xMx+fT27VZr7+iwLyqTJ8PChTGZki8hWahPPDH4/v5+u4aNHWvnVnm5/T2DVv/g+nbppVBfby2XHR12Lb7sMqssZO4H2LPHHisq4ODB9DIMfNyBA2cft2cPcKSDCu3gYHcV9E+hgg6gnz29V4JARUk7B3UG8043cVCvpqK/nQN6GZwqo0Jt8OyenivgYCkVJSc42HspNI6hosSSU/bsHw99ZbavuRT65ti+M6fZo1cASoUe46BcamUr7wCB+q4FTBvXCcDmrgVMG3MU6KP+4EymyUEQpf7QTKaVHqFDJ9DTNYbLxh6lVacAAt1KD2MYq+WMK+3m+Olx9CmUiNLfB2f6x1BGD/19wskz5UwrPUJrbyVrX7uCmjOvpavyb79tH9aZM+kPGmz/5s3p4zZvHnj7YMecOGGP48fbP0tXl207edJ+p6XFfqe31zLsensH/mfLQW1xqKA4TVV/mblRVX8pInPO+1XNAeDy0Pqs1LZsxzSJyBhgEpZwk1meR4BHwPo+RlKIWNVCwhJwwQ3399bUpP82K1fGICBCYhKhenoG36+ablbt7raA1NFhj11d6ePGjbN/69mz7doWdJL09Z27H9KTOVRUnL0Mwz+uvR042ktFfzvtveOhX6nQ47ZPK0CFiv5jtMskOHmS9v6JVGhqvUeo0A4QoZ1K6O6joqSD9jMToLufilJLkW4/NR5KyqmgI7VcRoWcgDN9tJdMtvL0HaG91KZsCgJtJ1OZ3WUBoaN3KrNLmqCrh86eKmaP6YauXjp7pzN7TDt9TEJ7FE6doq+/yj73HhDpp0/LKO87TXvvRZTQZ59rv9Knwjj66O0bQ1e3MK6knXbG03hoLIzdbR8iWPN/IFgOslGbm60/ECx4BcsDbc92TPBHOX7cAm/wzeX0aWuDP3zYHsPfoAYTNBGcp6GC4uRB9l10Qa9siTVzReQKLPjdCfx+xjF1wKeBl4HfA/4tl/2JkM5A/fa3reIlEpMM1ARccGPb3wuJykItK7Pr1UBE0klSQe0weLzoovR1rqvL/p27uqwbKRAsh/dDetq/oN8R0l1Nwz1u0iRg0nSOH59+7r6LMn5v2bVM2jDA82Qem4N9E1uga5rlhVS0QNe060Pbrz/rmNKm1Ic1dyqlDZxl7Fj7LMq77LOW1N+itAf6xlZQWgLjpoyna1ot5UD1R66G1StIqqHGKdaLyB9lbhSRzwFbLuSFVbUXuBd4DtgFPKGqO0XkqyJyW+qw7wGXiMhe4C+A+y7kNQdz6pQllN16q53kBT0Hqs93WvgSMv0e2M1ZBlNSYhfmkhL7+/X0WE0t+HjGj7dzrqMDamvtsazMKhHHj2ff39EBV10FM2fa8mWX2XqwL/O4yy6z5Zkzzz5usOfI3Nffn5vnGe6+8Hv4zd/Mvn2wz6uszJbLyy0oTp6c/oIR/E16emz54ovteaqqYjROO0+GGpIxDfgJ0EM6CNYCZdjwiOa8l3CEzid1fPXqc/sVg/XVq3NavNzIUdp/oYvd3yXsgQfgV6mRTGfOWOfVVVdZUlRmMk4WcRuSEfvs02Huy9XzDHdfeHmg9zbY59XcbNvCn3lnp31/VrW/UW+v1diXLLHuuIJvhTlPwz2nBg2KoSe7CViQWt2pqv92geXLm/MJirEaBwd2wd29237CmadXXjmsC25cxHYMaaYgIWrVqmF/aYlbUHSu0OVknKKIjAM+D7wL+CXwvVSzZ1GJXQZqQjJPwWoaGzem+3pjFxATkBDlXDEZqk/xn7Dm0l8CHwG+kfcSRSB2c6AmYM7ToJZYXg633WbjyU6ejLpU58HnQXUuVoYKiler6idU9R+w7M+lo1CmURdkoFZW2sU4mF5v7doCTbZJwIW2KO51mZCEKOeKyVBB8deDQoqx2TSspsZqhhUVcO21tt7WVoBZqAm50MY68zSQoAxU54rFUEHxWhHpSP10AjXBsoh0DPG7sROL2klCLrTV1ekxvYGC7ufNJjwPakODTcnX05Oe+9E5V3AGTbRR1dLB9hebWMyDmjnhdKDI5jyN7UxDYeFM4IQkRjkXd0PNaJMoschCLaIhFwOJ9Xyn2XgGqnOxMVTzaaKEs1APHYKf/tSuZc3NBdavWMSCrNO2NguA115r/byxme80mwQkRjlXLDwohgRZqD09dh9LgA9+0LrtCi7hpkjFol93JBKSGOVcsfCgmKGmxm4587GPwUc+YtMlxf7CHCNFkXUalpDEKOeKhQfFLIruwhwjRZF1GuYZqM7FSiSJNiIyBfgXYA6wD7hdVdsyjlkIPAxUAH3A11T1X0ajfLFIuClC27fb7dZ+/nO45BJLrhk3LoZZp2GegepcrERVU7wPeEFV5wIvkP2WUKeAT6nqNcAtwLdEZLD7O+ZM7KZ9KwJBgk1ZmfXjgvXrdnfHcL7TbBIwNZ9zxSCqoLgCm1eV1ON/yjxAVd9U1YbU8kHgMFA1GoWL3bRvRSCcYDNjhvXnfuxjlpcS+4AInoHqXExEFRSnqeqh1HIzMG2wg0VkMXYPx18NsH+ViNSLSH1ra2tOChibad+KRFH343oGqnOxkbegKCLPi8iOLD8rwsep3dBxwJs6isgM4PvAH6pqf7ZjVPURVa1V1dqqqtxVJotueEABK7oEmzDPQHUuNvKWaKOqNw+0T0RaRGSGqh5KBb3DAxxXATwLfFlVX8lTUQcUi2nfisD27TZBwvPPF1GCTVhCpuZzrhhENc1bHfBp4MHU41OZB4hIGfATYI2q/mh0i2c8CzX/ggSbykpLsNm61RJsli8vkgQbSGegdnbCww/DPff4NG/OFaio+hQfBJaLSANwc2odEakVke+mjrkdu3/jZ0Rka+pn4WgWcqBp31pavF8xVwZKsJk2rUgCYtj69fDSS95s6lwBiyQoqupRVf2gqs5V1ZtV9Vhqe72qfi61/H9VdayqLgz9jOqI5yALtbv77Gnfyso84SZXijrBJsyHZDgXCz6jzRBqaixZ0Kd9y4+iTrAJ8yEZzsWCB8VhSExtZhRt3w6rV1sf4oYN8OabRTxRgg/JcC42PCgOQ2JqM6Mk8/ZQCxbAzp22vbKyiBJsAj4kw7nY8KA4DJnTvjU0WO1m61ar7Xjf4shkjv+cOxeWLbOhGKtXF1lAhHMnBX/mGZ8U3LkC5UFxGDKnfduxA665xme5OV+Ja46+/3547DH7ueMO65i+886zJwt3zhUED4rDVFNjtZiFC61WM2+ez3JzvhLbHO0ZqM4VPA+KI5S4Wk6OBbPXPPusjfs8dKhIk2uy8QxU5wqeB8URyqzlNDfDc8/B6697/+JQggSb8vKzbw/V01OEyTWZPAPVuVjwoDhCmbPcbNxot5VavNj7F4eSqNlrMnkGqnOx4EFxhMJJN5s22a2lli3zQf3Dkeim53AGavCj6hmozhWYqCYEj7WaGvsJ7qJREvpqkZiL/Ahs325fFF57zcYjLlpktUNISIIN+KTgzsWE1xQvQLh/sbnZxi7+6Efw1lvehBoID9S//nprat6wIWEJNmE+KbhzBS2SoCgiU0Tk5yLSkHqsHOTYChFpEpG/H80yDkfQv/jmm/CLX8Dx4zBmDMyc6X2Lgcx+xBtvtCbnV18t0tlrBuNDMpwreFHVFO8DXlDVucALqfWBPAC8OCqlGqGgf/HgQejthcmT4X3vsxlavG/RZPYjTp8OH/4wXHddkc5eMxgfkuFcwYuqT3EFsCy1/E/ABuAvMw8SkfcC04CfAbWjVLYRqamB3/gNWLrU+xbDvB8xw0BDMm66yfsWnSsgUdUUp6nqodRyMxb4ziIiJcA3gS8O9WQiskpE6kWkvrW1NbclHYbMsYstLTZ28bXXkjl20fsRs/AhGc7FQt6Coog8LyI7svysCB+nqgpiwxSjAAALB0lEQVRolqe4B1inqk1DvZaqPqKqtapaW1VVlaN3MHyZYxc3bLBAcP31yRy76P2IWfiQDOdiIW/Np6p680D7RKRFRGao6iERmQEcznLYDcD7ReQeYAJQJiInVHWw/sdIBH2La9fCU09ZALjuunRLGdi+pASCYKhKIOhHbGqymnMi+ZAM52Ihqj7FOuDTwIOpx6cyD1DVPwiWReQzQG0hBsSAj11M9yO+/rr1I4a/GCSyHzGbYEjG/Plw661Rl8Y5lyGqPsUHgeUi0gDcnFpHRGpF5LsRlSknkjo3argfcfFiaz7euDHh/YiZfEiGcwUvkqCoqkdV9YOqOldVb1bVY6nt9ar6uSzHP6aq945+SUcuqXOjZvYjLltmzcibNiW4HzGTD8lwruD5jDY5lrS5UbdvtxrwD35gOSMtLbZ92jTrR1y0KIHjEbPxu2Q4Fws+92keDNS/2NICb7xhg/3BapVxDhZBk2llJVx2mTUb/+IX8Fu/ZUHR+xFDBhuS4X2LzhUMrynmUbh/saXFAkZ7uwWQYmhKDTeZzp9vIwxELPB7P2KGYEhGQ4PVEBsafEiGcwXIa4p5tHKlBT6wQCFi18H58y2QQDyHagRZpj/4gQX4q6+21sAbboBdu6wmfNNNcPfd8XtveRMMyairg+98B+6802uIzhUgD4p5FB6/ePCgBZD589PdSl1dNq6xsdFqlXFoTh2syXT6dGsVvOmmBI9HHExm9qlP8eZcwfHm0zyrqbEA8Qd/AAsXpgNiSwu8+CKUlVm/Y1yaU73J9AJ49qlzBc+D4igJD9Xo77d5UcEGuJeUQHc37NkDn/504Y1nDDJMP/tZePJJq+FCusl00iSrCfvQi0F49qlzseBBcZSEh2o0NVkQXLrUro3NzfDyy1brUi2sWmN4UP6sWdY8+uKLVmaw8i9caDVhH3oxCJ8Q3LlY8D7FURQM1QALIG1ttrx7N4wbZ8uTJ1vgbG2FL3zBbksVRX9jkEzz5JN2/V60yGq0ixbZhOevv27jENvb7X3cfffolS2WwhOCh23d6gk3zhUQD4oRCWemHj9ufYvd3RZ0mpthxw67cfHSpema42g1TYaTacCu5eHxh0uX2h0vmposYHuW6TDcf79PBu5cDHhQjEg4MxUsWSUIOhs2WK3s0kvP7W9csSI/tcagZtjYCG+9ZZmllZVWcz192mqyu3ZZ+caNs3J4hukI+WTgzhW8SIKiiEwB/gWYA+wDblfVtizHVQPfBS7H7rn4UVXdN2oFzbOgOTWoNZaVWRLO4cMwZoxdO4P+xvLydH/jl78MM2dal9SFNK0GgXDrVnj7bViwAK68El55BY4dsynq3v3u9OsfP55OFvLm0hHy4RjOxUJUNcX7gBdU9UERuS+1/pdZjlsDfE1Vfy4iE4D+0SzkaAnXGhsbrYY4c2a61hjub+zpgb17LXB++MPw5pvwyU/CFVdYwsuCBdb0Gox9DK+XlVmNtLvbgtw771gQbGuz7Tt2WCC89FILgLt327ytN9xgfYgiVnv05tLzEB6OceyYT+/mXIGKKiiuAJallv8J2EBGUBSRq4ExqvpzAFUt6tz1cBJOOOMzs79x1670MLfDh+2+hSJ2bEMDrFkDS5bAu95lAXPNGgtqEybYHTvA+gRfe83u3jFrlj1OmmRDLXbtshrqf/yHPX9/vwXQq67y4RbnbaDhGF5bdK7gRDUkY5qqHkotNwPTshwzDzguImtF5HUR+VsRKc32ZCKySkTqRaS+tbU1X2UeNeHhG3B2f2Mwl+qkSems1UmTLLAdOGA1vYMHrS/y4EFbP3DA+iQrKuxnz550pWX37nRAHDfOnn/aNHjPe6zG2NTk4w8vmA/HcC428lZTFJHngelZdn05vKKqKiKa5bgxwPuB64BGrA/yM8D3Mg9U1UeARwBqa2uzPVfsDNTfWFZmAXDRovStqbq6LLC1t9t6EDgz1ysq0tsnTYJTp2z5+uut37C727a3tUFpKfzd33kgzIlt2ywIPv20ddKOHWvbfTiGcwUnb0FRVW8eaJ+ItIjIDFU9JCIzgMNZDmsCtqrqW6nfeRJYQpagWMwy+xsXLbK+wPLydMBTTTetHj9ufY9gAS68fvp0evv8+dZfGfQhXnONNcUGU7h5v2EO3X+/TwTuXExE1adYB3waeDD1+FSWYzYDk0WkSlVbgQ8A9aNXxMIR7m+EdNZoZaUFvQULoKrKapCNjXbXiv5+G1bR2GgBb8IEm4kGLCGnrMz6HWfOtCbSefPgvvs8EOaFZ546FxtRBcUHgSdE5G5gP3A7gIjUAp9X1c+pap+IfBF4QUQE2AJ8J6LyFpTMpJygFjl3Lnz84+ls03nzrPk1WL/xxnT26YwZ8LWveRAcFZ556lxsiGpRdMH9Wm1trdbXJ7JC6QpRZ6e1f0+ZYm3e3d0WGL/5zUFriyKyRVVrR7Gkg/LzysXdcM8pnxDcuXzyzFPnYsWDonP5FGSePvOMDSTdt88yo7ZujbpkzrksfO5T5/LJM0+dixWvKTqXT5mZp35TYecKmgdF5/IpnHnqfYnOFTwPis7ly0Bznnpt0bmC5UHRuXzxzFPnYseDonP5sm2bZZru25f+8cxT5wqaZ586ly/33x91CZxzI+Q1Reeccy7Fg6JzzjmX4kHROeecS/Gg6JxzzqV4UHTOOedSiu7WUSLSit2jMUpTgSMRlyHfiv09Rv3+ZqtqVYSvf5YCOK+i/nuMhmJ/j1G/v2GdU0UXFAuBiNQX0r3w8qHY32Oxv7+4ScLfo9jfY1zenzefOueccykeFJ1zzrkUD4r58UjUBRgFxf4ei/39xU0S/h7F/h5j8f68T9E555xL8Zqic845l+JB0TnnnEvxoJgnIvKfRWSniPSLSMGnIQ+XiNwiIntEZK+I3Bd1eXJNRB4VkcMisiPqsriz+TkVT3E7pzwo5s8OYCXwYtQFyRURKQUeAj4CXA3cJSJXR1uqnHsMuCXqQris/JyKp8eI0TnlQTFPVHWXqu6Juhw5thjYq6pvqWoP8DiwIuIy5ZSqvggci7oc7lx+TsVT3M4pD4puJGYC74TWm1LbnHPnx8+pAjMm6gLEmYg8D0zPsuvLqvrUaJfHubjzc8pFzYPiBVDVm6Muwyg7AFweWp+V2uZcTvg55edU1Lz51I3EZmCuiFwhImXAnUBdxGVyLs78nCowHhTzREQ+LiJNwA3AsyLyXNRlulCq2gvcCzwH7AKeUNWd0ZYqt0Tkh8DLwFUi0iQid0ddJmf8nIqnuJ1TPs2bc845l+I1Reeccy7Fg6JzzjmX4kHROeecS/Gg6JxzzqV4UHTOOedSPCi6c4hIn4hsFZEdIvKvIjI+tX26iDwuIr8SkS0isk5E5qX2/UxEjovIM9GW3rnC4+dUfHhQdNmcVtWFqroA6AE+LyIC/ATYoKpXqup7gS8B01K/87fAJ6MprnMFz8+pmPCg6IbyEvAu4CbgjKr+n2CHqm5T1ZdSyy8AndEU0blY8XOqgHlQdAMSkTHYfd5+CSwAtkRbIufizc+pwudB0WVzkYhsBeqBRuB7EZfHubjzcyom/C4ZLpvTqrowvEFEdgK/F1F5nIs7P6diwmuKbrj+DSgXkVXBBhGpEZH3R1gm5+LMz6kC5EHRDYvazPEfB25OpY/vBP4H0AwgIi8B/wp8MDUT/oejK61zhc/PqcLkd8lwzjnnUrym6JxzzqV4UHTOOedSPCg655xzKR4UnXPOuRQPis4551yKB0XnnHMuxYOic845l/L/AV0GcoBaXoWUAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff3a5093510>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(7,3))\n",
    "\n",
    "ax[0].scatter(X_spca[y==0, 0], X_spca[y==0, 1],\n",
    "             color='red', marker='^', alpha=.5)\n",
    "ax[0].scatter(X_spca[y==1, 0], X_spca[y==1, 1],\n",
    "             color='blue', marker='o', alpha=.5)\n",
    "ax[1].scatter(X_spca[y==0, 0], np.zeros((50, 1))+0.02,\n",
    "             color='red', marker='^', alpha=.5)\n",
    "ax[1].scatter(X_spca[y==1, 0], np.zeros((50, 1))-0.02,\n",
    "             color='blue', marker='o', alpha=.5)\n",
    "\n",
    "ax[0].set_xlabel('PC1')\n",
    "ax[0].set_ylabel('PC2')\n",
    "ax[1].set_ylim([-1, 1])\n",
    "ax[1].set_yticks([])\n",
    "ax[1].set_xlabel('PC1')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "很显然，用标准PCA降维后的数据无法用线性分类器处理。还要注意右边子图中的蓝线我们故意下调了一点点，红线上调了一点点，这是为了清楚地观察他们的重合部分。\n",
    "\n",
    "现在我们尝试rbf_kernel_pca，看看能不能解决非线性数据：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [],
   "source": [
    "from matplotlib.ticker import FormatStrFormatter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_kpca = rbf_kernel_pca(X, gamma=15, n_components=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnUAAAEKCAYAAAB0a6PdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3XuwHOV55/Hfo7slkDiAEFhCgAUyN8sEH2OSjTEEsLE3gkSFE5KsiwR5VYRQXsfFlvESHC3ghCiwIZSxs6yNA6l1sI3lIAMxZTDYbMVcBIgDQjaSuIgjBAhJSIeL7s/+8XYzfUYz58ylp3u65/upmpqed96ZeWc06nnOe3lec3cBAACg2Mbk3QAAAAC0j6AOAACgBAjqAAAASoCgDgAAoAQI6gAAAEqAoA4AAKAECOoAAABKgKAOAACgBAjqAAAASmBc3g3I0sEHH+xHHnlk3s0AkJHHH3/8DXefnnc70sD5C+g9zZ7DeiqoO/LII7V8+fK8mwEgI2b2Ut5tSAvnL6D3NHsOY/gVAACgBAjqAAAASoCgDgAAoAQI6gAAAEog16DOzM4xs1+b2Rozu7zG/aeZ2RNmttvMzq+6b4+ZrYguy7JrNQAAQPfJbfWrmY2VdJOksyUNSnrMzJa5+7OJausk/amky2o8xbvuflLHGwoAAFAAeaY0OUXSGnd/XpLM7HZJ50l6L6hz9xej+/bm0UAAAICiyHP4daaklxO3B6OyRk0ys+Vm9rCZ/V66TQMAACiWIicfPsLd15vZByT9zMyedve11ZXMbJGkRZI0e/bsrNsIAACQiTx76tZLOjxxe1ZU1hB3Xx9dPy/pQUm/Uafeze7e7+7906eXYrcgAACAfeQZ1D0m6RgzO8rMJki6QFJDq1jNrM/MJkbHB0v6T0rMxQMAAOg1uQV17r5b0qWS7pW0StL33X2lmV1lZudKkpl91MwGJX1W0v82s5XRw4+TtNzMnpL0gKRrq1bNAgAA9JRc59S5+z2S7qkq+2ri+DGFYdnqx/2HpA91vIEAAAAFwY4SAAAAJVDk1a9IGhqSvvlN6ZJLpP32y7s1KIGBAWnpUmndOmn2bGnBAmnevLxbBQCoh6CuiGoFcA88ID30kHTccdL8+QR5aEp1AHfiidKyZVJfnzRrlrRli3TFFdLMmdLOnQR5ANCNGH4tgqEhackS6a23wu04gHvggcr9d98tzZ0r3XVXqFddB6hjYEC67roQuMUB3NVXS3v2hKBuzJgQyK1ZIz35ZKXOdddJd9whLV4sXXRRuB4YyPvdAEDvIqgrgmSAVi+A27lT2n//cH333fvWiVUHiOg5AwPDA7FvfCMEb3EA19cn7dolDQ5WHrNqVeXrFdfZsycEf8lg8LrrCOwAIC8Edd2uOoi7557aAdyhh4b6hx4q3XKL9PbblTrJ3jp68HparV65++6Ttm8fXm/6dGnjxsrtrVvD9bRplbLBwRD8JYPBvr4wjAsAyB5BXbepNdQaB3Fvvx0CtloB3MSJled44YXKr/Shh1Z662r18qGnLF26b6/cQQdJK1YMrzdzpjR+fAj69u6VJkwIX59jj63U2bgxBH9J27dL//ZvDMcCQB4I6rpNraHWOIjbvj0EbGbh9sSJ0uuvSxs2SC++GC6PPhp+hTdtqtSJe+uqh2npres569YN722TpJNOCl+XOIDbskUaN0668soQ9A0OSiefLM2ZE75OcZ3x40PwF3v1VekXvwh1GI4FgOyx+rWbVPekvfNOCL7iXrhNm8Iv6iOPSIdH2+Z++MPh1/bKK8Ptq6+W1q4Nxy++WHnuRx4JXSvJXr677pLOOENyZ6VsCdVKSTJ7dgi2+voq9SZNks4+O5TFdRcuDCtbzz+//vNdeWVYIbtlSwgUn3wy1Dv55EovoBQewypZAOg8grpukuxJ27xZ+vGPQ5dJHJwdcUS4JIO4avXKly2TfvSjSoCY7MFzH54OBYUXz51LpiS57jrp3HPDV0EKgdjWreG+yy4bPfCaN2/fOnPnVgK9nTul006TZsyo3D9tWrgPANB5BHXdYGhIuuGGMLSa7EnbvFm6/vp0es+eeioEb8neO6nSgxf3Dp5xBr11JZCcOydVrp95JgRwyR63uFeuFclAb/HiECDGXntNeuIJaceOcB957QCgswjqusEDD4Tuk6lTw6+sNLwnLY3es9F68OLewbReD7laty700CXFvWa1etzSsGBB6A2UwvTPX/wiHJ92WqWnsJEeQQBAa1gokbd4Ht348dLLL0urV1cWPbjvuyyxE69dPc8uXhVLTrvCqM49N2FCJQ1JbOvWyt8MnTBvXgja+vrCep2pU6VPfEI67DDSnQBAFuipy1s8j+6Tn5Reeil0d2TVUxa/dq15dvPn77v1GLpSrflz69eHvwnmzBk+d27hws62Je4FjHsKxyT+bGR+HQB0Fj11eRqtp6zTkvPsqnsHyWlXGLVyz33gA2GBdJySpK8v26HP2bOz7ykEgF5HT10ehoZCCpEjjhi5p6zT6s2zk8Jcu+RKXObada168+cGB8NQbB6S8+vinsK1a0OgedFFlRQrzK8DgPTQU5eHeFjzxz+u31OWp7x7ENGUbuwVS86vGxwMK2DNwlw/EhMDQGfQU5e15LDmW2+ll7IkTSPNtTv9dBIV56hWQuFavWJZzJ8bTXW6k4kT902xQmJiAEgPPXVZK8JWXSPNtUtuY4ZMxQsitmwZ3tslDe8Vy3r+XCNqbU/GwgkASBc9dVmqN6zZbQl/6821GxoK0QKJinNRL6Hw0qWhJ6ybgrhqtbYny3uIGADKhp66LI00rFkERehlLLEi93YtWBCCui1bwvbF8fGCBXm3DADKg6AuK0ND0te/HoKhblsY0QgWT+SuGxdENKp64cTOndLkyWF3vMWLWTABAGlg+DUrDzwQeuYuuKCYqUFGS1SMjuvWBRGNihdOJJMlH3IIW4gBQFroqctCGRL5jrR4Apmo7u3qxgURjaiVLJktxACgfbn21JnZOZL+UdJYSd9y92ur7j9N0g2S5km6wN3vSNx3oaS/im5e4+63ZtPqFiTnohU1ke9IiYqlSkJlUp2kolbqkrinq2hBXLV6yZKLMDcQALpZbj11ZjZW0k2SPi3peEl/ZGbHV1VbJ+lPJX236rEHSvprSR+TdIqkvzazPnWjXpmLRqqT1NRLXVKWeWdFnhsIAN0sz+HXUyStcffn3X2npNslnZes4O4vuvuApL1Vj/2UpJ+6+2Z33yLpp5LOyaLRTSv6itdGlGF4uYuUfXiSlbAA0Bl5BnUzJb2cuD0YlXX6sdnqhblopDpJVZFTlzSiLHMDAaDblH71q5ktkrRIkmbnMb7zxS+We65ZURIqF0gvJOpNzg2M5w/ecMPw+YMAgObk2VO3XtLhiduzorJUH+vuN7t7v7v3T58+vaWGtqXsc816YXg5Y700PFn2+YMAkKU8g7rHJB1jZkeZ2QRJF0ha1uBj75X0STPrixZIfDIq6y69MNesF4aXM9ZLw5Nlnz8IAFnKbfjV3Xeb2aUKwdhYSbe4+0ozu0rScndfZmYflfQjSX2S5pvZ/3T3E9x9s5ldrRAYStJV7r45lzcykjKkMhnNaKlOJNKdjKDMqUsaQXoTAEhPrsmH3f0ed5/r7nPc/WtR2VfdfVl0/Ji7z3L3Ke5+kLufkHjsLe5+dHT5Tl7voa5eSWXSiLIPQbeIoUfSmwBAmthRolOYaxb0whB0ixh67K35gwDQaQR1ncJcs4B0J3WVPXVJI3pp/iAAdFrpU5rkppG5ZmVHupMR9ULqkkb0yvxBAOg0euo6YWhIWrKEoUaGoEfE0ONwAwPS4sXSRReF616aWwgAaSCo6wQWBgQMQY+IoccKFo0AQPsYfk1b9cKAXh5qZAh6VAw9BslFI1LleulSPh8AaBQ9dWljYQDQNBaNAED7COrSRG665vTI3EPmio2OfHUA0D6CujSxMKA5PTD3kLlijWHRCAC0j6AuTSwMaFyPJCUmwXBjWDQCAO1joUSaWBjQuF7YF1fsbdoMFo0AQHvoqUtTj8wRa1sPzT1krljzmIMIAK0hqEtTD8wRS0UPzT1krlhzmIMIAK0jqEtLj8wRS0UPzT1krlhzmIMIAK1jTl1aemSOWCp6bO4hc8UaxxxEAGgdPXVp6KE5Ypko6NxE5oK1jzmIANA6gro09NAcsUwUcG4ic8HSwRxEAGgdQV0aemiOWMcVdG4ic8HSwRxEAGgdc+rS0GNzxDqqoHMTmQuWHuYgAkBr6KlD9yjw3ETmggEA8kZQl5aCTu7vKgWem8hcsPSx8AQAmkNQl5YCTu7vOgWem8hcsHSx8AQAmsecujRUT+4/4wxpv/3yblXxFHxuInPB0pNceCJVrpcu5TMGgHroqUtDcnJ/QYYLgW62bl1YaJLEwhMAGFmuQZ2ZnWNmvzazNWZ2eY37J5rZ96L7HzGzI6PyI83sXTNbEV3+Keu2v6fAk/sLqQvmLjLXq/NYeAIAzcstqDOzsZJukvRpScdL+iMzO76q2kJJW9z9aEn/IOnvEvetdfeTosvFmTS6lgJP7i+knOcuMtcrGyw8AYDm5dlTd4qkNe7+vLvvlHS7pPOq6pwn6dbo+A5JZ5qZZdjG0RV4cn/hdEFiYpIMZ4OFJwDQvDwXSsyU9HLi9qCkj9Wr4+67zWyrpIOi+44ysyclbZP0V+7+UIfbW1vBJ/cXShckJibJcHZYeAIAzSnqQokNkma7+29I+pKk75rZ1FoVzWyRmS03s+UbN27MtJFIUZfMXWSuFwCgW+UZ1K2XdHji9qyorGYdMxsnaZqkTe6+w903SZK7Py5praS5tV7E3W929353758+fXrKbwGZ6ZK5i8z1AgB0qzyDusckHWNmR5nZBEkXSFpWVWeZpAuj4/Ml/czd3cymRwstZGYfkHSMpOczavdwXbAasyd0ydxF5npli5XGANC43ObURXPkLpV0r6Sxkm5x95VmdpWk5e6+TNK3Jf2Lma2RtFkh8JOk0yRdZWa7JO2VdLG7b87+XaiyGvO44wqx8XxhddHcReZ6ZSNeadzXN3ylMUE0ANSW644S7n6PpHuqyr6aON4u6bM1HvdDST/seANHw04S3WVoSPrmN6VLLmnr32FgIKxmXbcuzJVbsIAgIg/sKgEAzSnqQonuwE4S3SWFHHbkoese7CoBAM0hqGtVl6zGRCSlHHbkoeserDQGgOYQ1LWqS1ZjIpJSrym9Q92DlcYA0ByCulZ1yWpMKNVeU3qHugcrjQGgObkulCi0LlqN2fNG6jVtckXyggVhDp0Ueui2bg29QwsXptxmNISVxgDQOHrqUHwp9prSOwQAKCp66lB8Kfea0jsEACgigrp2pJQXDUB95A0EgMYw/NqOFPKiAaiPvIEA0DiCulallBcNQH3kDQSAxhHUtYrdJICOI28gADSOoK4V7CYBZIK8gQDQOIK6VrCbBJAJdpUAgMYR1LWC3SSATJA3EAAaR0qTVrCbBJAZ8gYCQGPoqQMAACgBgjoAAIASIKgDAAAoAebUtYNtwoCOY5swAGgMPXXtYJswoKPYJgwAGkdQ1yq2CQM6jm3CAKBxowZ1ZjbVzObUKO/tARC2CQM6jm3CAKBxIwZ1ZvYHkn4l6YdmttLMPpq4+5872bCuxjZhQCbYJgwAGjdaT93/kPQRdz9J0p9J+hcz+/3oPutoy7oZ24QBmWCbsA567jlpzhzpt35L+u3fls48M+yK85nPVK7XrJGuvlq65hppw4ZwfPXVlePq8rfeCn/0xvclbyfvX7IkPG7Jks6USd1XHt+X/GyS4sfROYA2jLb6day7b5Akd3/UzM6QdJeZHS7J231xMztH0j9KGivpW+5+bdX9EyXdJukjkjZJ+kN3fzG67yuSFkraI+kL7n5vu+1pWHKbsKQVK6T58zNrBlB28TZhydWvCxey+jUVV1xR2ebQLPxxeuml0urVlesrrpCefz7U375dujc6ze7YUTlOlp90Ujg3LlsWbn/4w8Nvx/c/9JC0bVs4lx53XPpl8+dXFrJ1S7kU7kt+Nsnfi/hxyfpAk8y9fmxmZv8h6XPuvjZRtr+kf5P02+4+seUXNhsr6TlJZ0salPSYpD9y92cTdS6RNM/dLzazCyT9vrv/oZkdL+lfJZ0i6f2S7pM01933jPSa/f39vnz58labDKBgzOxxd+/Pux1pSPX89dxzUn9/6BWKfwPGRX/jH320tHZtuF6/Xpo5MwR98eiEewjqJk0Kxzt3ShMmSGPHhmjbLAQzUiX6jpcrz5sX6k6ZIt13n3TWWaENZpWys88OvVatlu3YIS1eHC4TJlReZ+fOfcvr1U+rPH7d668Pn9UXviA9/nj8DyrdeGNIhzU0FP562W+/8Hlcfz1psiCp+XPYaD11f66qYVZ3H4p62P6ghfYlnSJpjbs/L0lmdruk8yQ9m6hznqTF0fEdkr5uZhaV3+7uOyS9YGZrouf7ZZttAoDyu+KK0MOWtHt3uH7llRCAbNwYApVNm6TJk8NkxmnTQhC1ebN04IGhXlx+wAGV4G3s2HDf00+H6zFjwuMGBqTp06VDDpF27ZLeeEN6/fVw3/TpoWzjxlAmVeo1UzZpknTTTSGY2rat8jq1yuvVT6s8ft0HHgifw7p1lc9m3bpQHvfuxQvvNm+ulANNGi2oe1vSDElrqspPkfRwm689U9LLiduDkj5Wr4677zazrZIOisofrnrszDbbAwDl99xzYbh09+5KL13Stm0hENm8OfQ4bd0aJjS+804lINmxI/QuSaF83Djp4IOll14KZR/8YLj+9a8rt93DkO7u3SFQPPhgadWq8Nzu6ZQ991yYI/jDH4ZesoGBSvlv/mYoP/PM4eVx/Xrl1c8zWnn185x+epg/EL/vuAfujTekH/0o9NjVWnh3xhkt99bVS9g9MBDy5T/8cPjYTj01vI1nngl1J0wI8fWOHZVO2WQH7c6d4flOPDE8ZsUK6c03Qzx/0knplB966MhtePXVSt3DDtu3LH78hg3165mF2/F1/LjNm8NX9N13w0e///6VekNDlY7tcePCP6dZ+Htl797wN9LOnaHdO3aEst27pT17wrEU6h95ZJg6ef75Lf3Tjmq0oO4GSV+pUb4tuq/r/5Qws0WSFknS7LSXzLGjBNBx7CiRslq9dNV27Qq/RHuiGS3xMOc771TqxMfx0Oxrr4XruL5UuR3/GsYB2OTJ4Zc07mmLfzlnzw7PY4kBogMOCGXxa41Utn17GPrdtStc79kTfpl37AhfpF27wvXevSFwTdavV/7UU82VDwwMf91XXgnvc+PGEBSPS/zsrltX6d2rtfCuhd66OGF3X9/whN3nnivdemtY+7L//qHuvfdKP/5xiDv320/6+c9D+YknSr+Mxr1OOEFauTIcn3ZaiFNvuy1M/Xv55RDUbN4cmnzbbSF+X7++Ur5rVyg/9lhpcLBSvmPHvuXr14ePcdKk2m3Yvj18fFOmVOpKtcvGjw9fs+p6Y8aEr84BB4S/V6ZPD3Ueeyx8bQ85JAR+cYd1X1/4DM3CZc+eyj/X3r3h/Y0bV/mq7xlhEpi79MIL0qJF4XYnArvRgroZ7v70vg3zp83syDZfe72kwxO3Z0VlteoMmtk4SdMUFkw08ti4rTdLulkKc1LabPNwTGwFOqreD9RllxHYtezxx8Mvzwjzqd/7Zdq1q9KdEffSSZWy+HjPnvArHT/ulVfCdfxLt3595fnefTdcDw6GX8+kMWP2zWHTTJkUfrGnTAldLlOmVFaTvvrq8PI48Kyun1Z5/Lq/+lW43rhxeP34s/vlL6XDD09t4V0yYbdUuf7618NHP3Wq9L73hbKdO8M/WfzPM3VquH7sMWnGjHC8fHnlOO54nTo1BFkzZoTnevdd6dlnQ6C0atXw8pUrQ/mzzw4vX7Vq3/KNG8PzH3JI7TZs2xbaPHNmpW78PmqVvf/9+5aNHx+CxjffrMTPu3aFf5bJk0MH6pQpoXzv3vCae/eGr38cxE2YEO53D8+3fXulx64RQ0Ph3yOPoO6AEe57X5uv/ZikY8zsKIWA7AJJf1xVZ5mkCxXmyp0v6Wfu7ma2TNJ3zex/KSyUOEbSo222pznVO0q00VUOoLZ6P1BLlxLUtSxezYpSWrcu/AGUNG1aCNymTBmezDseQozj4zio27ZNOuKIcDw0VDlO1lu3rlI+aVKoN3t26L1Llm/bVru8Vv34b434cdVtiNsb15UqnbzJxyf/5qiut2dPCOa2bg3vI+603rkznF/iKaLx43bsCAHbnj2Vx0+aFALTMWOG/60z0t9JSXv3VgLptI2Wp265mf3X6kIz+7ykx9t5YXffLelSSfdKWiXp++6+0syuMrNzo2rflnRQtBDiS5Iujx67UtL3FRZV/ETSX4y28jV17CgBdBw7SgDNqZewe+bMEMwkR97jkeBp08Jl+/ZwiYOd7dvDT1x8HNfbtm14QBTX27atUj8unzq1dnmt+vHodL02jBsX6sR14/rJsvh2/N6q602aVJmvFy/iHjs29L7F5bt3V54zHhUfOzYEdWPHhsBu3LhKkBcHlclZAyMZMyb8e3TCaEHdFyX9mZk9aGbXR5efK+SH+2/tvri73+Puc919jrt/LSr7qrsvi463u/tn3f1odz8lXikb3fe16HEfdPd/b7ctTWFHCSAT7CgBNKdewu5LLw3Dmtu2hemQ77xTyUQzc2aYC7dtW7h89KOV4/7+yvEHPxiGNLdtC/Pc3n03DGO++650/PGh/LjjhpefcEIoP/744eXHHbdv+eTJoU312jBhQqiTrFuvbMqU2vXiwPaAAyrZeCZPDoHjO++E9S07doTPcsyYEFzGI/tjxoTh1vhx48eH4dh4EUejQd3++4d/j04YMU/de5VC0uETo5sr3f1nnWlOZ6WW52nZsrBqKe4blsKqrwULmFsHpCg5p27atBDQbdnS+Jw68tShF7H6tTyrX5s9h42WfHiSpIslHS3paUnfjoZNCym1k+LVV4fknNXmzJGuvLL95wfwnnZWvxLUASiytJMP3yppl6SHJH1a0nEKQ7K9jcANyMy8eSyKAIBGjBbUHe/uH5IkM/u2sl5hCgAAgIaMtlBiV3xQ5GFXAACAshutp+7DZrYtOjZJ74tumyR396kdbR0AAAAaMmJQ5+5js2pIIbFNGNBxbBMGAI0ZbfgVI4m3CSPxMNARcUqTLVuGbxM2MJB3ywCg+xDUtap6mzASDwOpS24TNmZM5Xjp0rxbBgDdh6CuVWwTBnQc24QBQOMI6lrBNmFAJtgmDAAaR1DXiriXLt7pd+JEeuuADqi3j+WCBXm3DAC6D0FdK556KmwA9+KLlYt72MAOQGrmzQv7vPb1SYOD4brRfV8BoNeMlqcOtbBNGJAZtgkDgMbQUwcAAFACBHUAAAAlwPBru9hVAugYdpMAgMbRU9cudpUAOoLdJACgOQR17WBXiXwMDUlLluTyeQ8MSIsXSxddFK4JMDqH3SQAoDkEde1gV4l85NQ7Ss9RtthNAgCaQ1DXKnaVyEeOvaP0HGWL3SQAoDkEda1iV4l85Ng7Ss9RtthNAgCaQ1DXKnaVyF7OvaP0HGWL3SQAoDmkNGkVu0pkb6Te0fnzO/7yCxaEOXRS6KHbujX0HC1c2PGX7lnsJgEAjculp87MDjSzn5rZ6ui6r069C6M6q83swkT5g2b2azNbEV0Oya71yE3OvaP0HAEAullePXWXS7rf3a81s8uj219OVjCzAyX9taR+SS7pcTNb5u5boip/4u7Ls2w0ctYFvaP0HAEAulVeQd15kk6Pjm+V9KCqgjpJn5L0U3ffLElm9lNJ50j612ya2AR2leiMgnyu7HrQGXyuANCcvBZKzHD3DdHxq5Jm1KgzU9LLiduDUVnsO9HQ65VmZvVeyMwWmdlyM1u+cePGthteE7tKdEYBPldy13UGnysANK9jQZ2Z3Wdmz9S4nJes5+6uMLzajD9x9w9J+nh0+Vy9iu5+s7v3u3v/9OnTm34fo2JXic4oyOdK7rrO4HMFgOZ1LKhz97Pc/cQalzslvWZmh0lSdP16jadYL+nwxO1ZUZncPb4ekvRdSad06n2Mil0lOqMgnyu56zqDzxUAmpfX8OsySfFq1gsl3Vmjzr2SPmlmfdHq2E9KutfMxpnZwZJkZuMl/a6kZzJo877YVaIzCvS5kruuM/hcAaB5eQV110o628xWSzorui0z6zezb0lStEDiakmPRZerorKJCsHdgKQVCr13/yf7tyB2leiUAn2u7HrQGXyuANC8XFa/uvsmSWfWKF8u6fOJ27dIuqWqztuSPtLpNjYkmTctacWKTJLhllaBPtc4d11ylebChazSbBefKwA0z8I6hd7Q39/vy5eT2g7oFWb2uLv3592ONHD+AnpPs+cw9n4FAAAoAfZ+Rf4KkmS4WSTPBQBkiZ66tAwNSUuWdOUKza5XgCTDzSJ5bnsGBqTFi6WLLgrXfG4AMDqCurSUMDDJREGSDDeL5LmtIyAGgNYQ1KWhpIFJJgqSZLhZJM9tHQExALSGoC4NJQ1MOq5ASYabRfLc1hEQA0BrCOraVeLApOMKlGS4WSTPbR0BMQC0hqCuXSUOTDoumWQ4vriHJMMFFyfP7euTBgfD9WWXsfq1EQTEANAaUpq0q0C7H3SdK6/MuwUdNW8eQVwr2E0CAFpDUNeukgcmQB4IiAGgeQR1yE5Jkww3i6TEAIBOYE5d2khCXB+5/MjBNgqSDgNA6wjq0kbgUhu5/CSRg20kBLwA0B6CujQRuNRHLj9J5GAbCQEvALSHoC5NBC61kcvvPeRgq4+AFwDaQ1CXFgKX+sjl9x5ysNVHwAsA7SGoSwuBS30lTjLcLJIS10fACwDtIaVJWkhCXB+5/IYhB1ttJB0GgPYQ1KWFwKWCfHQtIX8dAS8AtIPh107p5Xx1pHVpWq+n8yA/HQC0j6CuU3o1sCGtS0t6OZ1Hrwe0AJAWgrpO6OXAhrQuLenldB69HNACQJoI6jqhVwMb0rq0rJfTefRyQAsAacolqDOzA83sp2a2Orruq1PvJ2b2ppndVVV+lJk9YmZrzOx7ZjYhm5Y3oJcDG9LfvbZ/AAANhklEQVS6tKyX03n0ckALAGnKq6fuckn3u/sxku6Pbtfy95I+V6P87yT9g7sfLWmLpIUdaWUrejmwIR9dy3o5f10vB7QAkKa8UpqcJ+n06PhWSQ9K+nJ1JXe/38xOT5aZmUn6HUl/nHj8Yknf7EhLm9XL+epI69KWXk3nQX46AEhHXkHdDHffEB2/KmlGE489SNKb7r47uj0oaWa9yma2SNIiSZqdxXhOdWCTzNlWNuSjy0xZc9hVv68vfrEc7wsA8tCx4Vczu8/MnqlxOS9Zz91dkneqHe5+s7v3u3v/9OnTO/Uy9ZU5tUmZ31sXKWvKj7K+LwDIS8eCOnc/y91PrHG5U9JrZnaYJEXXrzfx1JskHWBmcS/jLEnr0219Ssqc2qTM763LlDXlR1nfFwDkJa+FEsskXRgdXyjpzkYfGPXsPSDp/FYen6kypzYp83vrMmVN+VHW9wUAeckrqLtW0tlmtlrSWdFtmVm/mX0rrmRmD0n6gaQzzWzQzD4V3fVlSV8yszUKc+y+nWnrG1Hm1CZlfm9dqKwpP8r6vgAgL7kEde6+yd3PdPdjomHazVH5cnf/fKLex919uru/z91nufu9Ufnz7n6Kux/t7p919x15vI8RlTm1SZnfWxcqa8qPsr4vAMgLO0p0Splytg0NSUuWVHriyvTeCqBeDjtJWrxYuuiicF20BQa9nJsPADohr5Qm5VcrZ1ucAuStt4qVAiRe5XrccSHXHvnoMledwy5eOdrXN3zlaBGCorKmZwGAvNFTl6UipgBhlWtXKurKUdKYAEDnENRlpajBEatcu1JRV44WNRgFgCIgqMtKEYMjVrl2raKuHC1qMAoARUBQl4WiBkescu1aRV05WtRgFACKgKAuC/WCo7vvHr6qNG+sci2MkVaODgx036rYuE0rVkgPPiitXl2sYBQAioDVr1lIBkdJd94ZAql4VWneWOVaKNUrYqXuXBWbbNO8edLkydIzz0hvvy2ddJK0cCGrXwEgDQR1WaiX3uSyy6TDDgtDsWeckW+ak+qFHHm3By1JLkSQKtdLl+YXOFW3ae5cafr0cHvx4nzaBABlxPBrXrpt4US3tQct6caFCN3YJgAoI3rq8lBr4cTSpWHC0V/+ZTY9ZHEi5EsuCUPDtRZy0FtXOLNnhyHXuFdMqixEyDLpb/K1nn9e2rFDOuaYfdsEAEgPPXV5qLVwYt06admy7HrIkomQWeVaGvVWxZ54YnZJf6sTDL///dIvf8niCADoNIK6PFSvKl29Wnr5ZWnChGxSnVTPn3v0UVa5lkS9VbHPPJNd0t/qBMNz50qnniqtX88erwDQSQy/5qF64cSyZSGgO+II6aWXQg/Z6adXhkfbGQJNDrPGz5OcP7d5s/Sxj0nXXNP6a6Cr1FoVe8MNodcsadq0ELcvXtz+kGxyuPWJJ8JXKunoo6VJk6Rbbmn+uQEAjaGnLm/1EhPfc086+8RW7zdb1ETIaEutpL9r1kgvvND+kGz1cOvEidIvfiG9+mqlDnPoAKDzCOryVms+29tvS9/5zvB9YqsTA9dSXafWfrPMn+tJtebarVwZ5tpVD8l+4xujJy9OJjj+whekPXsqz3PyyaHOk08yhw4AskRQl7dauzZs2CC99trw9CLVPW7SvkFcdZ1aaUrYJaIn1Zprd9RR0pw5w+tt3y7dd9++vXd33FEJ4i6+WLriikqd11+Xnn46fGUlacYM6bTTwleOOXQAkB3m1OWten5dnJT4wAPD7Tjdidm+iYGTO0CcfvrwXrn+/trDrNdfT5qSHlU9127x4n3Tn6xYIR100PDkxW+8IV19dfiKzZol3XuvtG1bOB4zRjrkEOnNN6VVq0JAJ4X5c+edR3JhAMgSPXXdpl66k5deGt7jVj20es89w3vlbrqJYVaMqNaQ7KZNYeuupMFBadeuyvBq/DVbtSrcf+yx4fGvv85wKwDkiaCu29RLd7JrV7g/uZAi/nWN5+Ale+V+8pNwP8OsqKPWkOzZZ4detqSNG8O2XrF4d4h44cWhh4a5eYccwnArAOSJ4dduM1K6E6mykOKWWyoz0rdvD8sY49sTJ0of+lDoKpk/P7u2o3Cqh2TjlaxSCN62bpXGj5dmzqzUOfZY6ec/l6ZODT1zW7dK48ZJN95IIAcAeaKnrtvVWtjwyithrCseWt28OSw/fOQReuXQllq9d1deGYK2eJh24sSwwOLkk+mZA4BuQk9dt6vuuZPCrPW1a0PwJoUEYLNnh1/aWvWBJtRKXjx37vB9Y//mbwjiAKDb5BLUmdmBkr4n6UhJL0r6A3ffUqPeTySdKun/ufvvJsr/WdInJMXpVP/U3XunW4rADRmrFegBALpLXsOvl0u6392PkXR/dLuWv5f0uTr3/Xd3Pym69E5ABwAAUENeQd15km6Njm+V9Hu1Krn7/ZKGsmoUAABAUeUV1M1w9w3R8auSZrTwHF8zswEz+wczm5hi2wAAAAqnY3PqzOw+SYfWuOuK5A13dzPzJp/+KwrB4ARJN0v6sqSr6rRjkaRFkjSbHcUBAEBJdSyoc/ez6t1nZq+Z2WHuvsHMDpP0epPPHffy7TCz70i6bIS6NysEfurv7282eAQAACiEvIZfl0m6MDq+UNKdzTw4CgRlZqYwH++ZVFsHAABQMHkFdddKOtvMVks6K7otM+s3s2/FlczsIUk/kHSmmQ2a2aeiu/6vmT0t6WlJB0u6JtPWAwAAdJlc8tS5+yZJZ9YoXy7p84nbH6/z+N/pXOsAAACKh23CAAAASoCgDgAAoAQI6gAAAEqAoA4AAKAECOoAAABKgKAOAACgBMy9dzZZMLONkl5qsPrBkt7oYHPaRfvaQ/vaU5T2HeHu0/NuTBo4f2WK9rWH9rUn2b6mzmE9FdQ1w8yWu3t/3u2oh/a1h/a1h/Z1t25//7SvPbSvPWVuH8OvAAAAJUBQBwAAUAIEdfXdnHcDRkH72kP72kP7ulu3v3/a1x7a157Sto85dQAAACVATx0AAEAJ9HRQZ8GNZrbGzAbM7OQadSab2d1m9iszW2lm13ZT+6J6XzOzl83sraza1mT7PmJmT0f1bjQzy6h955jZr6PXvbzG/RPN7HvR/Y+Y2ZFZtKtE7TvNzJ4ws91mdn6WbWuwfV8ys2ej7+b9ZnZE1m3sNM5hmbWPc1g521e+c5i79+xF0mck/bskk3SqpEdq1Jks6YzoeIKkhyR9ulvaF9U7VdJhkt7qts8vqvdodL9F9Tv++UkaK2mtpA9E/25PSTq+qs4lkv4pOr5A0vcy/OzK0L4jJc2TdJuk8zP+7jXSvjMkTY6O/zzLzy/Dz4FzWDbt4xxWzvaV7hzW0z11ks6TdJsHD0s6wMwOS1Zw93fc/YHoeKekJyTN6pb2Re162N03ZNSmpFHbF92eGrXRFf7z/F4GbTtF0hp3fz76d7s9am91+2+Nju+QdGZWf4GXoX3u/qK7D0jam1Gbmm3fA+7+TnTzYWX3/zZLnMPawzmsh9tXxnNYrwd1MyW9nLg9GJXVZGYHSJov6f4OtyvWVPty0Ej7ZkblI9XphEbb9rIkuftuSVslHZRB24a9dqSI7ctTs+1bqNDDUjacw9rDOax1ZWhfnjpyDhvXZqN6hpmNk/Svkm509+fzbg+AxpjZf5HUL+kTebclT5zDgGJq5hzWcz11ZvYXZrbCzFZI2iDp8MTdsyStr/PQmyWtdvcburR9mWihfes1vMs4q/ewXo217XDpvR+8aZI2ZdC2Ya8dKWL78tRQ+8zsLElXSDrX3Xdk1LaO4hzWHs5hqSlD+/LUmXNYlhMDu+0i6T9r+CTZR+vUu0bSDyWN6cb2JepnPcm40c+vepLxZzJo2zhJz0s6SpVJqCdU1fkLDZ/E+/0MP7vCty9R95+V/STjRj6/31CYiHxMlm3L+HPgHJbN58c5rITtS9QtzTksszfQjZfoP+hN0Yf2tKT+xH0routZklzSKkkrosvnu6V90fEShfH4vdH14i5rX7+kZ6J6X1eU9DqD9n1G0nPR614RlV2l8BePJE2S9ANJa6KT9gcy/v4VvX0fjb5vbyv89b2yy9p3n6TXEv9vl2XZvow+A85h2bSPc1g521e6cxg7SgAAAJRAz82pAwAAKCOCOgAAgBIgqAMAACgBgjoAAIASIKgDAAAoAYI6FJ6Z7YmSiT5jZj8ws8lR+aFmdruZrTWzx83sHjObG933EzN708zuyrf1AHoZ5y+kiaAOZfCuu5/k7idK2inp4mjT6B9JetDd57j7RyR9RdKM6DF/L+lz+TQXAN7D+QupIahD2Twk6WhJZ0ja5e7/FN/h7k+5+0PR8f2ShvJpIgDUxPkLbSGoQ2lEewt+WiEz/ImSHs+3RQDQGM5fSANBHcrgfdHm3MslrZP07ZzbAwCN4vyF1IzLuwFACt5195OSBWa2UtL5ObUHABrF+QupoacOZfUzSRPNbFFcYGbzzOzjObYJABrB+QstIahDKbm7S/p9SWdFKQFWSvpbSa9Kkpk9JOkHks40s0Ez+1R+rQWACs5faJWF7w4AAACKjJ46AACAEiCoAwAAKAGCOgAAgBIgqAMAACgBgjoAAIASIKgDAAAoAYI6AACAEiCoAwAAKIH/D07UQZu2RtPCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff3a4240110>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(10,4))\n",
    "\n",
    "ax[0].scatter(X_kpca[y==0, 0], X_kpca[y==0, 1],\n",
    "             color='red', marker='^', alpha=.5)\n",
    "ax[0].scatter(X_kpca[y==1, 0], X_kpca[y==1, 1],\n",
    "             color='blue', marker='o', alpha=.5)\n",
    "ax[1].scatter(X_kpca[y==0, 0], np.zeros((50, 1))+0.02,\n",
    "             color='red', marker='^', alpha=.5)\n",
    "ax[1].scatter(X_kpca[y==1, 0], np.zeros((50, 1))-0.02,\n",
    "             color='blue', marker='o', alpha=.5)\n",
    "\n",
    "ax[0].set_xlabel('PC1')\n",
    "ax[0].set_ylabel('PC2')\n",
    "ax[1].set_ylim([-1, 1])\n",
    "ax[1].set_yticks([])\n",
    "ax[1].set_xlabel('PC1')\n",
    "ax[0].xaxis.set_major_formatter(FormatStrFormatter('%0.1f'))\n",
    "ax[1].xaxis.set_major_formatter(FormatStrFormatter('%0.1f'))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以看到经过核PCA处理后的数据线性可分了，所以可以直接用线性分类器来训练一个合适的模型。\n",
    "\n",
    "RBF核PCA的缺点就是需要人工设定$\\gamma$，这个值不是凭空瞎想的，而是要做实验确定，在第六章，我们会讨论调参的技巧。\n",
    "\n",
    "\n",
    "\n",
    "### 例2 分离同心圆数据\n",
    "\n",
    "\n",
    "上一节我们创建了一个半圆形的数据，然后可以用核PCA线性分离，我们再来看另一个有趣的非线性数据集："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import make_circles"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [],
   "source": [
    "X, y = make_circles(n_samples=1000, random_state=123, noise=.1, factor=.2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD8CAYAAACfF6SlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzsfXl0HNWZ71dauiVrly3bsmzLu8F4EcaWWRLisCTAS4AABvJyksxLcpjMDMmxz+HM5CXjGWZIXjI5yZAZo2HiCYYZ3pwJCchxg8TyEoslLDI2WMLgRbZlSdZura291ar3x8/f3NulqurqRVIv93eOTqu7a7ldVfe73/r7NF3XSUFBQUEhuZAy1wNQUFBQUJh9KOGvoKCgkIRQwl9BQUEhCaGEv4KCgkISQgl/BQUFhSSEEv4KCgoKSQgl/BUUFBSSEEr4KygoKCQhlPBXUFBQSEKkzfUArLBgwQJ9xYoVcz0MBQUFhbjCsWPHLum6XhRsu5gV/itWrKCjR4/O9TAUFBQU4gqapjU52U65fRQUFBSSEEr4KygoKCQhlPBXUFBQSEIo4a+goKCQhFDCX0FBQSEJoYS/goKCQhJCCX8FhWjC6yX66U+JhobmeiQKCrZQwl9BIVyYCfqaGqK33sJrLIxHQcECSvgrKIQLo6D3eomqqojWrSN66aXZF8JzufAoxB2U8FdQsIKdJm0m6GtqiIaHierr8TqbQniuFx6FuIMS/gqzj1h1TxjHZadJ19QQTUwQ5eTgtaoKf2NjRE1NeJ1NIWwcj9L+FYJACX+F2cdsuyesFhs7Yc+a9IoVRD/8IVFHR+B+VVVEixfj/eLFRAcOEPX3EzU2Es2fj9doaP9OFkqz8SjtXyEIlPBXmF1E6p4Ix2qwWmzMhD2Pq7oaGvSlS0RtbUQVFYH7TUwQud1473YTdXURnTxJNDBANDqK1/Z2ouPHI/tdThZKs/Eo7V8hCJTwV5hdROqeCNVqsFpsrIR9Tg409gMHiAoKiM6cISouJnr+eaH919UR6TrRhQvib8MGCN077yQqKiK66y6iNWuI9uwJ/3c5XSjNxqPrRLW1seleU4gJKOGvMHsIxz0ha8ThWA1Wi438OQt7HtfYGNw2LS1EU1NE8+YR+XxC+9+7l+iZZwL/HniAaONGaPtNTbAWOBbg1G1j/F1OF8rdu7H4PPFE4JjKy0N3r8VqPEYh6lDCX2H2EI57QtaIQ7UarBab9vbAz1nYaxre9/QQTU4SvfMOXvv7iTIziV5+2VooHjkCF8/77xOlpxMdO4YxejzO3Tby7/J6iQ4ehBY/MWG/UEZiNTg5lkJCIirCX9O0A5qmdWmadsLie03TtH/WNO2spmn1mqZtjcZ5FWIYZhqklXtC9otbafqVlUS/+12gIK+sRDCWtzUruDJbbCoqAj/v6YGGX1uLMZWWEq1cSbRwIdG99xLdfTdeN22yForl5URpadh31y6iG26ACyg7214AWy1QlZUQwi0tWJisFspIrQarY1VWEj32mLIAEhjR6uT1DBE9QUT/YfH97US09vLfDiJ68vKrQqKCNcgrryT64hfx2e7dRE8+SfTnfw6haIbqaqKnn0aWTUaGEGBHjxK1thLdfz/R+Dje+3xEJ04QbdmCRcR4PnmxkfHuu0TLlonPS0vxt3o1XDpEEHznzk3f9/hxcXwGa+nj41hIWFM/cAD+/+Jiot5eXBPjvtXVsBI+9zm8ZyH/b/8GC2X+fHyfkgKLwnh+WcjzOXbuNF9QPvtZ6+tuPNbx47i2ZWXTx6yQEIiK8Nd1/U1N01bYbHIXEf2Hrus6Eb2naVq+pmnFuq63R+P8CjEGozbKQsdsQTDud+AAhOgvfwmBzALs0iUIw7ffJkpNJTp1Cm6a0lJoqZo2/XwsyM3OE2wRMtuX9xsaCtyvpoaouRkCWtOgqa9YgYUjJwfbmAlgr5foRz/CdllZsDaIsKh1dOC9243r9cAD5ouOmZAfGbF2r1kJcvlYvIhNTWFRC7ZozASc3COFiDBbPv8SImqR3l+8/FkANE17SNO0o5qmHe3u7p6loSlEFV4v0Xe+gyCq0YcdzAddXQ1BuGgRhPv58xBcg4Pwuy9fDtfKihVE27dDKOzcCcHb1OTcxeHUr+2k6It/18QE3k9OQlN/+20ivx/aOJEQwHIAuKoKC1pmJv44YPvFL2Ihyc/HMawKxqzcWi++GNy9ZnZN+Fhs7aSm4rrOhf9fxR5mHDHVwF3X9f1EtJ+IaNu2bfocD0chHFRXY8Ju3473Rm1Udk/IWihr/S4XBJDPR9TQQFRSArqE/n5819gIYXjhAlFeHlIxe3pwDDkwaqWtWlklZpAtFXalmFkzExNEN98s9mtqwmerVuG97Do6dAhjKC2Fa2dyEr/r3Dlcu9tvh9srIwMLXUYG3FQ7dhA9/DAWCF2HVuz1mru11q8PtFxkLdoK7CJraMDi5XZD+Pt8zlxG0UQo90ghbMyW8G8lomXS+6WXP1NIJLS1QejMmwchvW4dhMjwMATa1VdjOzMBzVp/URHer1tH1N1NdOONRB98QHTVVRCoU1MQUGNjEP4ffgiXSWoqzrl+vbmLgwVgaan9IiRvLwsgq8XLKq5wxRXTXUdeL9EjjyAGsH8/0SefYNwjI7hOBw7gWF1d0Pz7+1Es1t8PgZyRgfNyfOOhhxCbCIZg7jYiMVaPB4tRaan4jrX/mfD9m7l3zOIYKu4QdcyW8PcQ0cOapv2aEOgdUP7+BERFBVFnJ1FhITTG2loEVtvbMYmN7omqKgiWP/9zCB2/P9C14fcT7duHY82bh8VA1yEwpqbgF9d1CM+sLLiK+BxmgVEWnHaLkLw9C6DOTgjmrVun72cVVzADH9PlguD3euHumZyEa+vcOaIXXkAAmwi/+/33cQ17elA4ZhXfIDIXpKFq0WaLmc8Hi8Ns30h988aFySqOobT/qCMqwl/TtP8iop1EtEDTtItE9LdElE5EpOv6vxJRNRHdQURniWiEiP5XNM6rEENoa4NgWr8eQnvnTgjyn/+c6PHHzTNnPB4IvSuvhKacnh74vc9H9PrrWEz6+yHkBwchnIgg6DdswLnshDALFJcLCwQLcatAqFEAcR3ANddM32/nTmfCTz7mhQuggJiYwLXSNLwfHMQYn3lGXB+XC+d//30sfl1d+G758ulasZmGH6oWbXYdPR64qHhfWeA7sSqCXRN5YbKrBVHaf1QRrWyfLwf5Xieiv4jGuRQoNjMhKiqEht7fj8WA3RRWmTPf/S7cGgcPEv3zP0//LUYXxPg4aBb8fgSFR0fxXTAKBRYoXi/2ZYuEYZU+aVYHYNzPLMXUbgxuNyyWjAwIdSL87/PBelm/XlyfqipQTLzxBtGCBeAOmpqCS8gY39D16YKUP4tEi+Y0Vr5PLKDfegvX/vDh6ec0sz6efJLoq18levZZ8Z3ZwmTlRjNLsVWICDEV8FVwiEi0rVBhXGja2oi+9a1AOgSvl+i11+DC6O+HgD12DEFfq0nL6ZEdHRDwZpqdURC0tMCa0HUI0dFRHMNpCmNpKRaQ3l5YJFYC0HheszoAPvYjjwQXfsZjlpYSLVkCbX7RIgSLeVy8kLFgHByEwM/IgPuJCFlAxviGrk8XpPyZnbst2CIg36f0dOzLAt+sjsFsMeTndXAQ10EOoBsXJrv7ohBVKOEfb5jtTAjjQlNRAcFeUSGCjTU14LUxBgnvucc6mHrwIDTq+fORw2+WT24UtHffDe1YXmSam0GtYCX8ucHK+fNYjJy4EZz68c00VytLwHhMo1VjHBcvFqdO4bd7vdC+iUSdAwv12lq4hIyCtKjIXIvmjCMz5UFe7HU98D719CBQXVwMV1xjY2Adg1k8gq2PFSsQz7j11vBrERSiCiX84w2zmQlhXGiuuAITfOlSuF/+4i8w6UM11VmbJHKuwdfUQHNcsYLoU58Snzc1gVrBCnV1CDifOQMNmt02kboRzAKTzz0HoXzddcEX5mDXLJRAsscDIW0UpDt2gP7COG7OODIbo7zY6zrui6aJrK2TJ+GKunAB1oic1cXbyvEItj4GB+Ha6u7G4vXjHyODS7l35gxK+McTZjsTwrjQ/OAHmMCZmZi0jz9O9A//EJqgIoJwqasDd46swb/1FoSL0R0hB2ybm5HqSUR0+jRcH3YCY/duoosXQVPAAehoXCuzwGR9Pdxia9eKeIcT68JpDMdqu1AW36oquJvy8lBDIY/RuNjn5cHV5naLtNOhIQSd09IQexgYgOWxeDG2XbQIx5ItATluceYMlIfeXmQv3X9/0EsdMmIxJhaDUMI/njCbmRDGhSYrC9rlunUQBFNTRP/5n/BR8zZO4XJhAdmyRQQ4m5rwuZnLhH/3rbcKd5KuIwPlwQedBVqd5PWHIjCMAndkRFQZnzkDn7bThdlpDMdqOzsai5/+VPwmrxf1Fu3t8OHPn48xbtuGQKyxBsLlAkEdu6beeAMCPj0dFtTy5fh89WrcS6Mbi6270VE8L2lpuEa9vUS5uURPPUV0xx3hC2irezabMbE4hqJ0jic4YcU0Ilx+duNCU18PDd3rxeTNyoIp//jjoR3X6yV69VVM/mPHoMUzbfErr1g3XTGyef7ud8Hpiq32ZSZQ4+8NhU7AyOm/di187GvWQNAxn79T2mmnvyXUXgbyb6qqQuzD70dtwdmzcOVUVGCbp58OvFavvILfwM9aaSlcbjt34jfv24dU2z17zJ/N3l78cdzi3Dm87+7GQtDVFRl9QzSprJMQSvOPJ4TqXiEKXwsyarbsamlpgSmfno7vX34Zrp9QxrNpEwSJHBRmv7VRQzezdsx8y1axBbN9mQnUyt0RqhuNF7RQsp2MYwxmmYQa6zH7TR4PhOHkJLbp6IBr5/RpZB8ZayA2bbIO2vOY+NkK9mxyai8zlN5xB56fyko8a7t3W9NxmGn3duSBqjrYEZTwT2SYTRCrVEQjrHLzH3kEmR5ut0hPNLJc8rZW1abGmMW2bdaxDOMi5PNN9y1bCWyrfRcsiK7AkBc0hl22k3yNnMRwwon1GH9TVRXu2aJFYp+hIWjg69fDUnFSA2Eck9MF04z5dP1688XYuJ+dKzAaVNZJCiX8ExmhpCI6QXU10ioLCoiuvdY+5sB0Cp98AmoAWcgGa64iHzfUFEkZdvsyX000BEa4hUlOYzihxnrMFgumyiaCwCeCe4qzd2680VkNhHFMLhe0+epq6+CtFfOp32++GBv3My4w0aSyTmIo4Z+osPJ3W/HCOIHHA02N3S5WaZN8brcbBUEsGJw2V2GYCc9wBe1MCoxw3HFEzn9LOKm0xt/U1YXMnJQU4QcfGsJ2Rtpppy4lpqro70fw9tOfJvrVr7DNnj3Tidpk5tOzZxF4Li9HnMSMPM7KIrNaDF98EQubSh91BCX84wWhZqNY+buJgvvKrc7vciE9kAjasxklA597eBhanZzVsXdv5Gl44QpaK4Hh8SB+MRcCw+lvMWMHtaNoNlsstmyZXqHM3cqYtnn9evNuYUbwtSRCZlNREdw4jz8ugq9yBzCz8bS04PNbbsF7o8Vl5+qyWgyNVNYKtlDCP14QauDWyt8drnuDfbapqaL4x2zx4Ek7NgaznqkIWPufqzS8UKiXYx3BrmGoiwoTtwVLm2XwtTxyBNZEdjaer1/9CsVjqamBFdtmi9d990GRkIvEZKvDztUVb/crRqGEfzwgnGyUSHzlZufnMn8+rxUlA2v9jY34nBuSsFtgrpp0BGP9jJeioGjTe4T7bHHwv7wcz9Krr8KNMzSEhAC7HgA1NXBDpadj8Xj7bTwzcsGeInibcSjhHw+IRvpaJJOJtX5NE8FCInPtv64OmSMDA9D8ieBn7uoSgd1YS8OLp6KgaKcyhns8WTP3euH+IcJ9X7IEioKsHMjMnlVVRLfdJjLGPvgAbLCy5eFEu4+nRTsGoYR/rCNalA6hNCQ3oq4OgoHJxQYGkB+uadMXDz7PuXOBx/D5oB1a+XjnCm1taKK+Y0dsjMcO0ab3iOR4sjLxwQcInGsaMnk4hVTW/mVmT9mdQ4RjXHll6L8lnhbtGISq8J1rBKvAtfN9RgqePHJTcTPs3Uv05ptoG/jjHxNdfz3RP/0TWiiaLSrG6tdnniF64AEwf87E74gEFRVYALq7Y2M8doj2sxDJ8fge79sHd2JODorc0tMh9AcGoDAcPx7oWnr11cCq4dpaWIi8KDj9LaqSN2IozX+uIPeUtdNeIkltNGugIX/Pk+fAAbhm7DQoztvXdbAxzlYu/EyCu48VF4fOxzMXiPY1jMbxuMDtC18QnxkL3Dwe4VrauFF8x3GDHTuEC0jmGrJz59i5q5Q7yBGU8J8rOBWmkaQ2GhtoyBO6qgqFNp/5DFw0aWlEf/d3oCQwErWZtUEM1d8cixkadt3HYtGNEO1rGI3jBVtA7FxLdkV/Zs8sI5i7SrmDHEG5feYCsjBtbIy+C4SPzw00Vq6cTpb29NMQeO+/L9JAGxsx8Yzgyt7mZjFmnnChmtvhEs1FG1bdxyYm7InyFAJh5uJ75hmxsNi5lszI4CYm4Bqyc+fYHVO2aK1I/BSISGn+cwNOh7x4UQjTFSui53LgySE30JA12qoqnLOwkOjjj+Hy8fux7W9+I5q0EGEyHTgArdjnQzremTNYUMIpnY8VraymJrTuYwrh4cgRMMJOTCAewDh+3LpZvBnBnww7a0NuZ3n8uD1vUJK7h5Twn23IRVBTU8iaiUSYWh3f2EBD9md7PDh3dzcWISJkakxNEfX1EX3lK2j1l50Nrf/CBUHn3NaGScwEYKH4h2e7BaUdYjEGkYgoL8e1NhaQGXsN8GdOso/s+hc88gj2Gx9HuunU1PR6FKfxNjskwMKh3D6zDdbKe3shfIaGRDekYNz8oRy/vV00/vb7Bb98VRW0ol27ILwzM7FddjZ83yMjcAVVV+N4Hg80fl0Hh//kJAJ0zOlulUJq5tqRg3RznVkTzF2hEDnsMnLMuPgjzWaS979wAQoNETLV+Hnm7bh/QbjZQmbjjzMo4T/bYI1z+XLwn5SVBTbIiFT4GBt/X7yIB/vUKXzOmRdEIl8/OxtUv1xmn5cHV097O77buhXj3bgR7++6y36cdk02jFqd8scmLqwWe6tFwa5ZkZNYEe/f0ID4zeSkaD504AD2jUa8LUHSTJXbZyZhZhrOtGYZ7PhM5nXkiHDnaBosg0uXhG+2sxPBX6ZqyMpCRpDLhYlk1X4vWJMNRbebHHCS5WP069s9ux5PcBeNzFXkcuGcr72GV+aXyshAgP/oUSg54cTbEqRhjNL8ZxKzbRo60Y727kVhzpo1RN/6Fvz77e0Idt51F9HDDxN9+9sov3/lFWQBcePu/n6QdnV2Wv8mK20vnBaUCnOLSDKzrBb7qirnFiCfv709NE2bn7XaWjy7o6NQcp5/HsdpacGxiRAP0zTn2n8CWbBK858pzEVw02kmjVWTF+Nk3bQJn61cOf0YZoFRO21P+dLjD5FkZlkF1Fkrd2IBGikh7DRt2cqWiefkArIPPoCgv3ABn3V2QvsPJXkhgSxYJfxnCrNtGjpZbLxeol/8QuTpj49jIjz3HLQrbs4hIxTK4wSaGEmPSJUXq2eG3Y4NDYhDXXGFeQ8BY63KHXfgczPef7PMHbnnwNtvo3ixqwvP+bx5SHRob4fQ37nT+TOeQFliSvjPBKJNwOUEThabmhpoXrm5COCePk3U2oqUOLcbTbQjeYATaGIkPWZKeXHaQ8BYq9LWhhoTM95/s0p5uefAmTPIaNuwAc/8//gfeEZHR7EI7NkT+vgTAEr4zwRmWwN2stjwNunp8Hl+8gmEckoK/P3hsCoakUATI6kx08pLMKtCPn9tLQT0sWN4VokgwEtKRP9lM9oR2fVTVgaf/E03IY5FFNiBzK7/cAJDBXxnAnV1EPYvvQTzdqaDm1wxfPSoMHW5qba8zcQE0ec+R3TDDZh4N9xAdPXVmAQrV+IYDz8cl8ErhShipphkOYBbXW1f7yGf/8Ybie69F8/qgw+CHba4GL58s0p5OQBrTD548UXzQLDHE97viPN5ojT/mcDevaG3xosER44QvfceHuapKTzgHR0Yw/33T9fkCgrgR735ZpTecxXwsmWgaU5STUjhMmbKfccumqkpaOlE5laF1flra1GVzrw9mhZYKX/yJJ77khI874cPB1ovvb1Ejz6KPzkQ3NsbvKeF8XfEAkVJhFDCfyYwU5k+ViXlmzaBAnf7dgh2TYNGPzSEoNb3vgcNp7gY27e3w49aX4/Pc3JQ2fvRRziuXR6/QuJjJtx3PCfcbrgcr7kGn5u5RK3OL/P+sBWt66JSvrMTLs2UFNCTuN3mjKGhuGSNcy6WKEoihHL7zARmisbAqnL2wAE8gC0tROfPg32Tz11RQfTOO7AM2AV16hT2bWjApLl4ERpVX58o8IrjsnWFGATPiYEBaOq1taHVe8jW6/g4nvOmJig0ZWUI9ubkIHtozRqi1avN60refdf889pac1dOVRVoINiFGksUJRFCaf6RwkoziHawzErjYOK1oiJkRjQ3wwSemEBV7v79MLE/+QRFXOwGMo75kUfA8kmEhcKsObuCghPYzYnSUuFq+fnPQ3O1cDzr5ZdRbLhihXCrcv1AaSkWhfJypJU6BbtpZVcOU59PTBA99RTRpz/tbG7HCelbVDR/TdNu0zTttKZpZzVN+57J93+iaVq3pmnHL/99KxrnjQkYtfGZCpaZaRys9btcoF7w+URBTHU1/PfDw/Ch5ubiAeZAmNWYL1xA+ic3Z1dQCBUzMSc4DvDOO7BYJybwjB45EnnVrRVXD1OfL1yI18cfd/Y74oT0LWLhr2laKhFVENHtRLSBiL6sadoGk02f03W97PLfryI9b0zA7KGxojGwMitDOY/x4a6uRuFKaiqoF/r68DC2taGQ5sMP4f8cGMCi0diIQK/VmJkQKz1dTCwFhVAQypwIJfuNaUkyMqDxFxVBuy8vjx4bqFGxevppnM/txuvBg4H9h81+RxyRvkXD7VNORGd1XT9PRKRp2q+J6C4i+iQKx45tmBXC2AWrjGZlqOcxPtwvvohGFTJ8PuRBp6Yi+yEzEw+o14sH+Be/gH+0uHj6mM1MZwWFUBDKnAgVsotzeBiZPi+9hPfhZidZKVY9PULrJ4LlPDYmXKdWiCPSt2gI/xIiapHeXySiHSbb3atp2o1EdIaI9ui63mKyTfwgFN9+pBkCVqlv69dPn1geD7pxHT6MTJ7hYZSzM31DayuCYcYx6/rsVyUrJBZmsjjM6OLMyoJwzshA8/gf/jC841opVk8/jcD00BDmUU8PznXokLXwn4vK/ggwWwHfF4nov3RdH9c07U+J6N+J6CbjRpqmPUREDxERLV++fJaGFiZCqeKNVBtwqjnJXcIWLoT7ZmgIXP2jo7AGCgrwIBvHbEbspnh5FELBTFa219QEujiJRCpzJPUHVm0m09LQZ4MILtC+PhRCXnGF/RjjaA5FQ/i3EtEy6f3Sy5/9N3Rd75He/oqIfmp2IF3X9xPRfiKibdu26VEY28zBaSHMbGoD/PD19IievHl5WAiWLUMq6LLLt0oet5wzrXh5FMLFTHI71dVNd3ESwYqNxK1k1WaSwZlwXDdjxwMUZ9xW0RD+7xPRWk3TVhKE/oNE9D/lDTRNK9Z1vf3y2zuJ6GQUzju3iCWmS04t83rx8JWWotCLtZVQWAsVFMLFTD5jfOxopVHKDLd27limkDh/Hha03byNszkWcbaPruuTRPQwEb1KEOq/0XX9Y03T/l7TtDsvb/ZdTdM+1jStjoi+S0R/Eul54waz0cSEU8vKy9EKct8+orVrwYOyZk1orIUKCrGMaKVRMsOtXBBpPKbsRm1qEgHmGM7gCQVR8fnrul5NRNWGz/5G+v9/E9H/jsa54g5W2gCTQ0VDgzEGk+Mo40BBwTGcsIE6sQq8XqRtjo+L1qVm7ljW+hsbiebPFwHmBJlPit5hrhBNDUbOUQ6lTZ6CQjwhGLWC0zlVUwONPzUV760audfVoWamrw+af1+fCDAnAJTwnwuEUghiRx9rFkw+cADaSrQrjBUU5hLBqnidzinezueD+9XvR2FjQ8N0d+zu3XCfbt0Ka2Dr1oRyoypun7lAKG4ZO/pYs2ByVxfR5CRS1WTEaMaBgjXq68Fc3NyMxmv33EO0efNcj2qOECxxwumc4u1ukjLNm5pwcc3ml+z2OXsWlfMJQnmuhP9sI5rFYWapZVu2RJ7+pjArsBPu9fVEP/sZFNOLFwXX3t69RPfdN7fjnhPYpVFyRy/jnNq2DVTncgyAGy29+KLoH8zHMQr/ujq4eQYGsH9nJ/blPhlxDiX8ZxrGIFSoxWF2aWZKwMctWLgXFBAtXQp38s9+hpTyzZuxKPj9RCdOIMZYVAQZ9Nhj0AWSzgKwe9Y9nsA5RQRXzuOPg+JEtppDabS0ezdWXqY5efllVP26XKE1f4lRKJ9/uAjWyk1uWScHoZymfpqlmVVWYvar4G3UUV+PBk/f+AZe6+tn9nyVlRD8BQXg3uP/KyvxfXMz5E5GBuiZNA31ej6f2EbhMoxz6sgRaOyVldNjAKHE24xst0QIEjc1JUQMTWn+4SJYK7fqalAoL1sGVwy7bYwajGwZGI9vTDPLzoYqWFam/PdhwMrNEkwLnwk0N+NcMvLy8DkRxvfee9D4GWNjeM/bKFyGPKe4Ijc7G02L3O7AGIBdbMBopRvZbtmyOH8+IfpdKM0/HATTHpiEqr8fZqddxo1VeprsbxwdhURqaIDJqVI3QwYL+L6+QAHPC4KdFm51PDtLIdj3y5fj1jI6OohefRUs3I8+KmTTRx9h3e/pgfAvKcG+Chawa+re3g6hfeFCYG4/zyXjXNy7F0WTDzyABvL33ku0caOgPI9z7V8JfycwuniC5RtXVyMrgAiCu6HBPN/ebhHZvRtpZbt2Ed19t0g3+8xnVOpmGLAT8Nz8TIashRtht5A4+Z4IVkdfn0gdf+MN9OEpL8fjUlGBuH16Oh6Tri4I/rQ07KtgAtlVyk3dz5yBz4xbmjY3g9nWmNtvNxcTtN+FEv5OIGsETvKNDxyAc1bTMFtBb6bRAAAgAElEQVQ/+EA8gLLQtltEZH/j+LioLpQ1mWDaf7C4RBLBTsC7XNC6Dx0iev11aOEDA9YadjBLwYklsXkzvBMFBejzk5uLpJXiYsim3FzImC99CdvOn080MjLdFTXbsYqYBs+Z3l7R1H1gABdY1zGHW1pwYY8dQ2vTjz7C92wx1NfjVZ6LRgtg1y68xnm/C+XzDwajRjAyYp+tww0nZHR3E735JqyApUuxXbCUTzmI1dwcmG62fr0zcrhgcYkkwvLl0LILCsRnAwO4DXV1uKxZWViz33iDaMECWPjf+Mb0NMxg/vrjx3GuwUF8fuWV0331cvyBCGGcRYvEuHJz8bp4Mf6mpuDJMAr+2Y5VxDR4zixfHrhyc+qzxwO3DzcrKinBnNy0KTC5Ii9vemp1nHH1O4ES/sFgDBB5PNAcrPKNf/ITSBBZygwNIUtgyRKhLVilfFZV4QHcs0c8VI89JtxIROLcdoVbkTaQSTDccw8EIxHm9sAA4nanT6PXzZo10Pjb2tDHvrMTnraioulC1WohWb4cArmxUWTnjI6i7ezGjSgWJQoU2unp0A1+/Wt8v3079uvvJ8rPDzy+ywXtngPWnZ04xsQEdAve5skn8Zd0sEsHNQrvggK0NL3lFljq+fn2HD7BUrTjpGm7DCX87WC22vf2Ev385+Y32OOBZbBypeDNJ8JicOYMik5YEFsVrRw6hPMac5NDRQKSu0Va8TpvHrR6TSO69loofp98AkE+Po75XliIyz9vnhDu/FpZifOZLSR9fUTf/Ca2ueoqoo8/hiKZkYFjnzhB9Fd/hd/w3e/Chz9vHqyDwkJsc/EiblleHtb/wUHctpIS/K9pGBsXfXV0EF1zDY6VkQFrYXSU6P/9P5wnKbV/KxiFd3s75uWlS7iA7e2Yu9nZgU1idu6EUB8ctOfqj0MrWwl/O4RSkMULxRe+AGG7ejWCttnZwtwM1teU09SKiyPT1hPMRK2vJ/qXfyH6/e+hmJWVTdfG6+sxR997TwjztDTchi9+EWttQQHRnXcKYd3SIsIhGRmQBa2t+KywEP7/oiJo5v39EL4cbB0dJXrtNbxmZcETV1kJWbB5MwTxqVM4V16eWED+1//CWP1+HM/tJtqwAZ6I1lYR3L3+eoyDz71uHbR6ueirt5foj3+E1ZKZieNrGq4RL1QJh3A1bFnZ8vng88/MhPn02c+igOuuuzBHx8dxcffsQTvUt94ieugh61aRcWplK+Fvh1A688ia9vHjmKVbtliXnls1joiGth5n7eTswC6S06chkIkg4K+/PjCI+oMf4JL390NJI4JQbmvDnL3uOqJVq/A5C+I334RgHhwEHZLfD+VP1+Hz7+nBI1BSAi1d04i+/30hoLOzIdyHhiBDOPmjsRHnYME/bx7G9fnPYzvOA/D5sHicPYvFY+lSKKKjo4g5btiAOEBfHyyWvDxR9EWEcfX24q+wEJbG2BismoStBQhXw5aVLY8HK2lpKd6fPo2b0daGGyG7YA8fDi7U49TKVsLfDqH2zl28GFpDTw8idAcPBg8Qmx2DKDJtPc7aydmBM2cmJqBNaxo+f/990cP7mWcCO1empGC7kRFcVk2De0dOzsjLw/YZGaJav68P2xYVIURz6RK+6+vD++uuQx7+xYsQts3NWDSmpnDMkRG81tdjDBwvuHSJ6NOfxquuiz9Nw+vAAKwEXhDy80Ws4PrrcRxNgxUgF32lpUHe9PVh4cvJQcwgIwPGY8IhWhq2cX6cOiVeZYqIQ4fw3k6ox7GVrYR/NCBr2qdPY6ampEA62AWI5crChx+GNOJZG4m2nkCcP5xZw8HTzEwI3MZGCMKxMbwfHYUwZEJTFqw9Pbj8HR346+nBwtHfj8vb3499UlOx3bJl0PqvvJLoN78RKeN8W8bHIeTT0nC7JifNxz01BXmRmooWyt3dgdvqeuD2aWl4DNga4AVnYABZhddei8VgYACfd3RgwfL7sUCtXo3jnDiBMf/oR9G/F3OOaGnYTuYHu2DZ3LQii4tjK1sJ/2jgyBHkCw8PQ6i73Zih4+N4QKwCxER4yL7zHczs5cudUTHHYWZBuODMmiuuIHr3XXzW0QFB3d8Pt4jXC9fN1BTWXNbE+b3bDYHq8UBgut0QmhMT2CYnB9pyfz9kyqc/DeGcno7tdB0B2JYWCFq/H/FAv9963OzP9/vxGLByaQWOKaSnYwHgBWZ4GOO+7Tair32N6Je/hMvJ5RILX2oqFsPCQlgNJSXTU0Ljnhp6tjVsK6FeUQHLgd1OcWxlK+EfDZSX4yFYtw6+iMWLiY4ehQ3e3m6vBVRX4/urr4Yks1soGHGYWRAuOLOmoADa7/HjkANr10IwFhZiXnZ0QGimp0PYTk0J37quQ6C/+y624YyeqSl8NzgoNPzubhR8jY5im9FRHJ8XGz6Hz2c/bk3DOPx+IazZMjHblgjHnZqCMOeawPR0COzmZlg9W7ZgIWlqwrFYNg0P4xqVlWE/RsLUAsy2hm0m1H0+oldeQZW9FVdXHEFV+EYK2Q/5yit4IGtrwenDlYVWbd/a2vDwzJsHtdJYWRjsfEnA8SNXwvp8RLffTvSVrxB96lNYY8fGsFYuWwYBS4TXlBT85ecT3XorfOfp6SIAOz4e6HrhgmwiuIY6O6F9M917Xx8EeUqKtatHxtQUjskLzMTEdFcPQ9ex3dgYtvP7hfWSloZYAQv7hgYsXjk5kH+Tk/jz+TDG48cD65vC4S2KSThlwzUi3Cp3ruqV/x54AAVhxop8+RxxVFWvNP9IIfshN22CfX74MJK9jxxB0RebqkY8/jgWgOXLMXvr6iC57EzZOM0siASbN5tXtpaUwNs2Pg6t+LOfRbrkypXQ0jMzBbVSV5fwqJllwug6bkFamsgW0jTh9nG58D2naNqB4w2MtDQIczs3kREs/MfHRRD7pZewGA0MYKHgRUPT8Dc4iMdJ5v6Rq425Ze3kpEhbjRvtP1wNO1Qr2cqlaud2ks/BNBJxYJUrzT8S2PXQvXQJM7Giwny/v/5rov/8T6hv/f2Y5c3NqOS10v6D8QolEebNQyEV5/OvWgXv27PPEv3ud5h3772HhaKnB4oiU70MD1sf1xiU9fvFHwtkI9i/X1CAcaWkiM9TUnCclDBn2tSUyCjiGEV/P6wSjmcQ4f+0NHgO5W5gjY34zSkpQmGenMRiZiSbSziEYyVbsezaVeTzOSor8fDFiVWuhH8kMD4QRJhdXi8qeouLiZ5/Hg5p437PPQd1LD0d6mlbG6RFQwPcRU7Ol4TN2Vnrd7tRsHXbbciH//u/B/XB5s245D/9qQictrcjPTM/H1ZBqGAXDmvORvcNp27efjuCsjk5EMScdWSXFRQM7L7iwLHXC4WUXT0pKXifkoLxDQ0Jgc7VxrqOR9DlEimsV18dp+6fUBCMfdcIJ8yeRrcTdxHLyYHy1tzs/HxzjMQV/uH63kLZT+4HykLb7wdpzNQU1ECfL1D793qR/z84iNlYVCQS2Jctg+q2Y4f1+cLxeyYQgvmw6+tBhcQBXA6epqVBY9Z1XO5QwG6eYG4bZgAoKcF7Pj8vGnysggLkBcjuI870YRcOf6ZpIm7Av4WpHjIysAgwK8Hy5Yhp/OAHRN/+NgzL1lakmnJVM1NOLF5sT1sd9wjHSrZbLMxiAPv2YQ6zwtfTg5XVrFdADCJxff7hZsSEsp+xH+jx40jFeP99fM+O55dfBrEL5wU3N8Mq0DQkaDc2YiZedRXRihXWKWxxnFkQLTQ3Q0i+/rqooF2/XgixykqRveP3i3x8IqzD588H99kboWng5OP6Pdb82aWTmop1vqQE8/6GG3BLOeuI00/T02GxcOqpUbjzK4+P3U4MPsb4uNiPx5GXh3GUliIO0tWFMXd04P/MTJGmOjqKgLbLlcCNYULNDgonlZTncmurCAbpOm6+U+bdOURiav7hZsTY7WdmERi337MnsOvP3XfjddOmwF4AnIs3OYn8/r4+zODOzqR05Vjh+efBjrF2LV6ffx4Ci9mxmcjstdeg8d99N9ETT8CDxtk1U1PCbcPuGdl3b/TFm71nTXnDBiiFXEGcmgotfsUKjOUPf8DiMjQE3aGwUCw+7P7hFFJeNGSrgAW9vADI4KA0/5a0NOyv64KZoKsLY+TYRnOzCA6Pj+ORKypCi4m+vgRuDBOqlcx8/kePikUj2Dw8ckT0B/j4Y8H1cepUXFjlian5h5sRY7efmUVgtr0xP9jnw8OwdKnI+bv5ZnHOQ4cgJZgF1AldcxLg+eeJ/vIvIVSLi2FE/eVfgsSMMTSEudfXJzh9RkdFgJU1aRmy1s7fs8BnTZ6RlQUZkJEBa35sDALX5cJ28+dDXnR0QMtevBivr78OTXt4WKSU8j4+X2AVsjGIzMqjXVooLxxM+ezz4bHq6oL84bRPOa3V64WROW8ezj8+Hoe5/qEgVCuZ26aeOYObsngxqvVLSgSzpzEDqLwcWj/3B7jnnrias4kn/MOtBLTbT9enc4rwZ8btjUVa7BYqL4dANxaObNkimk0o/DeeeAKCnwUcv9bXg3zx2DFY1z4fhNzICNbf9HShOZvl1rPGzq+ZmZi7fX0wvHw+aMYbN0KYtrVhUWEhnpGBPzkn3+XCZ1deiXO4XBDC8vnZ2ONFxyoAbCX0jZicxOKUmYnHiZPG3G5cl/R0yCO/H5/l5cH3v3On6EWQsILfDlapnLt3IyugrAxaxU03Ef3f/4v4m5niF8ecPozEE/7hVgLa7ccau6zh82d25zG6hayqd9mllAR0DWYwox9obYXGPzQktG6XC5p9Rgb24zx+1pZdLgjF7GwIQta2jVp+SooQzKmpYNXUNBhn2dm4XevWwZK/eBELC+fT8/kXLQIH0LlzELTXXYf5zzTQvN3wcKBAt0oXNUL2+5uBM3jWrBFVzv39ON/AACyLefPwO8fHIfzb2kTf4G9+M+zbFd+wiunJVnxnJ1K2N27Eg6lp08nk4pjTh5F4Pv9wM2Ks9qutNV/hjxwJfh5j9kBVlXkmkVVucRLAqtl5Xh7mYEsLhO/YGN5PTBC9/TaSq9jvzVo1F2lNTeE1N1fQO7C2zemRKSkQip2dEJjz52P7nh7EFM6cgea/YAGOkZEhisT8ftxSXihKS8XjMTCA18lJjDlcWPn9ZXDx1vbt0OQnJ0WMwu8X2Ud5ebhG3d1J4O6xg1VMz6jFj42JBu+cvulywdysrsY2CZB5l3iaf7juE6v9uBGLcYXfscO6uQORdQFYSoq5+bhiBY63fbt1RXACQk7dJMLrpUsQ+Owd46wYTYPr4vx53AafD0KYM3v6+0XmTVqaYMbMysJlJsL8HBuDts5UDbm5+D4/H8cfGYGWzIHjefOwjc8n0kUHBrBdaiq0/JdfhsfA5RJZvFZxB6ew24+tnYYGQT1RUIBz5+fjGrKVwb0Ibr8dj1ZSCn6i6TE6bplaWiq0+PFx0cbxzBloA0S40P39mMN33JEQbtrE0/yjjXBXeKsCMLc7UOvg7ewqghMYzc3QTBmdnUhVJBJrIPPuZGcLSuUFCxBcnT8fQpbpjbOzIfzS0zF/s7NFtgvTNnNlLAd6x8aEwM/IwHarVoFDKCsLgp+rYrOzBcf/0BAYfq+9VvQVOHUK33m9wVk/7cAWixXYhZWaClfZwAAeIa4qTk/Hd0xAt2oVQksJm9cfDFbKWE0N6nR4jtfWipJopmefmsJ8LyrCjWbtP86ReJp/tBHuCm/M+jl/HrOztxcqWk2N6PJVUIBWTVwR/PWvQyVOghiA3Ay9sxMa9OAgBG5RkWiQMjIi1tH0dMQEduzAZR4ZgdDOysIlXLcOvnf2d8vNU+SAK+fhj41BUE5MYDGZN0/EHg4exLH7+gQnzqJFGONNN2Hh+sMfcJz8fLzKAeJQwW4rJ7EBTUPcgyklOGOY/yYnMQZ2bVVWgt00KWGljK1dC63+0CEsCI89hkAOER4ur1ekbWVnY0X3eIjuv39OfkY0oYR/JLDj1ZcXDa+X6L77IOFKSkTcgLt8MRk9m6Pf/z58CXFADhUpmLL50iVo/Mx4kZ0tLsvoqNh+chLCt78fAdmiIqFlp6bikr75ptDw2Wpgbx1rwtyukShQ2J48CS1540a4R/buhTzIzhZB38xM3MY1a3Cu4WEcPy0NfvWpKeH24bx8tjJSU815gnhh0jQsfFzFG0z7X7IEsQm5RkDOJuLq5rExyLqCgiRt7m5UxpqbcdGamvAwVVTgRhuVPbmpC7uFenth9sW5YhYVt4+mabdpmnZa07SzmqZ9z+R7t6Zpz13+vlbTtBXROO+cgrtv1dQED9RWVYnWU2fOiDzEF1/E67FjmK2colJTIyp9Y7g8PBpgyubWVuF/LyoSmTKyhp6WBp98VhaMpv5+KGks0AcGYGANDEAbJhLuFebmkStpzcDsm9/7HigS1q0DWdzXvgZN/+tfx/vPfAbn6ejAK7tduMGK3y/ooNk9s3493FWpqfhs0aLAojIW1BMTwYngNA3XYeFCuKsYZkVsTAzncuHYCc3nYwWZnmHfPmj8d96Ji2LFwUUUPp9WHFA7Ryz8NU1LJaIKIrqdiDYQ0Zc1Tdtg2OybRNSn6/oaInqciP4h0vPOOaqq8AAYffhm8HiECjswIHj+16+fXhG8ZQt8CcwRkgQZQJs3Q9u+7z4QtU1MiNCKyyW46zMz4ZZZtw6+9fFx7M8K2egoLjPH7Kam8J2uC957O02aFwevF4rehx/CKiECadyBA4I87p57hCePNW2OLfAfZx9xn+CFC0WHMF4QmJSNtX4mgeOaQCtkZeG8IyNikUxJwedyMzj+zO3Gtv39Sez3Z7BA53ZsZhxcDCcxPzNBHwcZfNFw+5QT0Vld188TEWma9msiuouIPpG2uYuIHr38//NE9ISmaZquh5sHMcfweomefhpqakuLyPu14gzJySHatSvQbOSc/8ceEw+XzwcrIDMTjtodO+KucCRcsO9/8WJc1uFhzM+sLCRAnTiBz9evh6uluxuXaWLCPKWSfeeTk/DF9/cH96MzD8/ICI49OChI44xuks2b4fopKMAtZm2d44OcZ8/uYpcLbi3OSMrLg6wYHRWuIHYpsdVj5fZJScEiODgIl4/M8cPFa+z2kVNGmfXT7cYiFtdtHSMBEzIeOyYKJDIz0YyJObgYTmJ+xtoBOaW0shILxZ49MTeHoyH8S4ioRXp/kYiMtJT/vY2u65Oapg0Q0XwiuhSF888+2I2zcCFm09iYtZAOVgwiP1weD6REaan4jOsDmpoSOgDMvv/ubhRWcdes7dvBd1dYiEt26JDwb2dnY8Eww8SEyHbp7Q2kV7YCV/BOTaHwy+/HvOU0VCM6OiAz1q3D9ty5i9NNR0dFMVl6Oh4TFsQLF4om8nLbRRbgExN4FNi6kZGWBgNy40YsAD4fxqLrgt6CwX0AdF10/uJexHHd1jESMCGjca41NYVepGWsHeDmLpxSevw4NJeysuDHneXe3DGV6qlp2kOaph3VNO1od3f3XA/HHKz1c9VPdjYWAqsWjMHMRtlktNr20KGYNyEjxebN8HgdPgyBNDqKuXPmDASb1wtBW1gIBYuZMe166cpdt7ghSzD4/YIbx+dDtt8nn5g3PenvF1p4fr7Q2tmt43LBclm2DJ8zDTORKF7jTmHz5yMeMDqKR6GkBL9fJn5jTExgmyNHsFiuXYvAL7OAsiXAQWQijGPpUoxl9WpBiT0xgd/4ta/BGkjo5i4yotUW0qyQk1NKmQZ2fBxpY8H8/7PsKoqG5t9KRMuk90svf2a2zUVN09KIKI+IeowH0nV9PxHtJyLatm1b7LiE5BW5pga2dkoKZj+RYBjbt2+69h/MbJRNRrNtOdtg/vyELgKrryf6j//ATyspEUkYGRnwvff1Qah1dGCeckDXiSuHaDqhmxHsYmH3DVfIaho09sceI/rtbwP3yc9HsLW1FdsyYRuft6hIBHbb2mCB8PllOZCRIX4PjzMzEwKaBb0xiMtjZT6jzMxAYjruD8DVx2VloiKY6yo6O0Eqy0ZpUlkB0WgLyanaxtqBoiIEkU+fxuepqcGtCjMLYoa1/2ho/u8T0VpN01ZqmuYiogeJyGPYxkNEX7/8/31EdDiu/P3yilxXBxW1rEz8fepTkFhud2irthPq6SQpAqusFCRtOTlIdnK7odn298PNwa5ZXQ8UiKHw85sJfrcbgnLBAgjh9HRhJbDw93iI/uzPsEjV10NLPnlS5NCnpgqtnkhUBS9eDAHd1iYEttEC8flE8RrXHVy8KAxLDg7Lv4HTTjMy4FXgOgQuZps/H7+L+/oWFECol5UJCoqTJ4VFkp8fx83dow2rTB3jfK2unu7S7epCILmhATEFNu18PvvEkFC7jkUBEWv+l334DxPRq0SUSkQHdF3/WNO0vyeio7que4joKSJ6VtO0s0TUS1gg4gNOyNna2oi+9KXQA7TBqKf53MYisL/4i4TT/puboTCNjUHAZ2cjj767W4RWiPBderoIsmZmBjZHN2PLTE0Nnt7JLJdvvy3iDZoWyJ3/xz9CKLOv/tIlCG2/H8I2NVUEd0tKsO8VV6CtKzd/tzo/U1FkZgoK5kuXBDmd2QKnaVh4uDk7j5mre7OzMc6vfAWLFYMzmDizeHwcbR2JEry7l1M4IX/r7YVGkJ5uztK7Zcv0mMLZs0gPf+IJ583hZ1D7j4rPX9f1al3X1+m6vlrX9R9d/uxvLgt+0nV9TNf1Xbqur9F1vZwzg+ICTlbkigosAN3dzldtJ23mQklJi3MsXw6BOTYmgpYDA5hb+fmYA6xApadDq87MhEuluBjCO81EleEgr6bhFsr0B8wXlJKC3r6NjfCNs99cpmRmiuTXX4er5KOPRFaQrkN4s5uFuYI4vZO3sbJQOBU1O1uMh4U+8xYZFzX+rfy5rPVrGsbG9QZywxauqygoEL+fGUmJcM0TtruXEzglf1u8GDfsxz9Gl58nnhB1BHv3mscU2tuJ3n3XeXP4Gdb+YyrgG3OwE9BsGjY0wE4uLkZ0srAQ27S32xd5VFfDLGSJYLzhXi+E/PCwMB/llLQYLh4JB/fcA4F21VVwRXR3Q4CtW4d5MzQErZTbGBIhgJmXR/S5z0FguVzTj8vCm4OhLIQ5GJuRAWFbVwfBx4VWRi19bAy3RHY3+XwiMM2+fU7jXLMG2vTFi1gQFi40D94yOMaQloaaBx6XlbXCmUVEgR29eNxskdxyi3ma6qOPEv37vyN11u0W1kdCd/dyAitlz0pAV1QIlzDLhPZ2fC8vCPv24aH4whecN4efYYZQRe9gh2Ac/2+9hbZvPh+08v5+wQ5WUYGbakXR4PEgellbK7p4EYkOXjU1kFDr1kFKRZqSFuNgjbSyEpdvwwbE0JcswXr62mtwpy5bBkHq9SIuUFKC/5kWgqtYjeD0Sc6l50WAtWQi7Ds2Zp5BZBTCbIFMTgrmz5ER0TD96quRH0BE9C//gjiBlebPrhrZ2pF7BdtBZg7lGoPJSSxAO3YQ/fVfm/dL2Lw58Jrzd9/8ZhIEe43ghI6vftXa/WKkhyDCQ/Dqq0Q33oiL+Oyz2O7SJQRUgnX94+/miCFUi9W467Zt2/SjR4/O7SBkkicZS5dCNXW5iH79a8x2jhKOjyOqdv48OACGhqbHCTiDJzs7+PdvvokFgLkCGAne/evb30aWz8QEBFlREebTpUswsq69FgFYIszZzk5ov5z3zn17iYSQd7mEX56LwMwWCidISwukYCguxjg3bsRte+cduJF4YSgogBVgFZNg9tHJSZHGygLdrBsZs3oytTWR2C4tDfrCrl3Q/D0enD8vD4tLX1+SZPQ4BXfb27JFtGVk2LVnZLr30lII8w8/FOXcX/oSbv7Pf44bY8YPZBY/jEKuv6Zpx3Rd3xZsO6X52yEYxz/nzW3eDPuZCA9LSQlmn1UgN1igV/5+48a46w0aKerriX7/e8wVbtJeV4fLkZODNo5yVerKlRD+k5P4fnhYaPNpaaLatqAAl3t0NLJGK0Qi4yYjA7d7yxZoz0ePCmYO7jLGKZlMI81xCM7YcbmExeL3i7HbxQjkcbAFwi0bV63CZ2439JeNGwP7JRCZVy0nJWQf/yuvYB7L2j2ReT9t2SU8Po59+KEaG4PrJyfHedc/hlWweQaghH+okG96bS18CMeOiSiiz4eH6NZbsb0xch8ssp8AvUEjRWUlsmeIhKDs6oJSlp0N2udjx4h+9CNs098veukWF8PzNjws9ufg5uQkFgFOdYwEnAFUUkL0p38KZa2wEHKA+/sS4bGQaZrlauHhYSwe7LaanBQBWw7gpqUFWguy1s/1CDJ//5o1+I1cJ+DzweJYu1YcQ2X0SJAVrU2bnCtaskv4xAlRxMHpah9+CK6ul14SpE68qHD1YEmJeXbfLOX6K+EfKuSbfuON+KypCYxk3BXolVesV/lgdA8J0Bs0UjQ3w3P23nt439oqSNOWLsVnZ8/Cku7sxJxh9s/hYUHwxhw3XV3Yh7XuaIC179xc6AC8WA0N4XaxwGbtnkiMJSUFGUw5OWKs7LdnigneXtdhRDLXT0qK6FRGhN8uLzAdHfjdzNtfVAQPpYykz+hhhKNosVuGe2R+8gn8/uPjYmWenIRvrakJN7m8HCYYg91MOwwsOME8AlGGyvYJFcEoGDwe+8i9vH9Dg6B1Nvs+TnuDRorlyyEAr78eSpRMo5CTg89SU3G5NQ1B4KVLRfB2agoC0kjvwDn74Ya5uD8u+9oLCzHPX34Zi0BfX2CfAAbz98iB2aEhzG8+HruB5AAua/NMx5yejt9fWAg5lZmJRWT5ciiXbje2W7BA0GKUlGA/rpBWGT0SwkmxZLfMjh3I4lm7VvB3c64tp4F98IE5A6jTVNIZpnRXmr8ZnDZpkbd/5BH4HMwCuFb7swbw4INzHvmPJTDJW0EBjKuPPxaNSxjczJAvHfsAACAASURBVL29HYvDggW45FlZEIBtbUIRY5ZNXgjCEf7cP3hoCPvn5gp6p8lJcBJxtk2w47NVkJZmTcSWni60e471e73YPi0N4ygsRObgkSPwMra3oxawsxPX4O23kXW0dy88E0md0WMGswweInMfP9F0wb1tG9LQ2F2bloYHIzsbq/Q990yfz1ba/RxY/Er4myHUoEs45toccHnEC4wpiKWlSH3kXPbeXrguuHuWz4e0UA56cj5/erqo/s3NDfTFO4VM0zw0JJrG8+cffxzYstEuN18GN1jhZvK8UPG+ctex8XHR00DTsADk5YkFiIvBTp/GAjU4CDdRayvR3/4t+iTcd19ovzspEKqiZZznFRUIEGdnI7By1VUo9rDK4vnFL5AC5jSVlMh6IYoClPA3IlShHG6Adpb9e/EEY176HXfAwu7uhtDv7YXLY8ECwavDXPUcFxgfF+0T5SAs9+E1E9CcNikLYblq1uXCQjI0hP97esRCQ+Rc8DPkIi1O7zQWl7HbZ2AAAt/lwvvVq5FV1NeHtFfOMuSGN9z68sQJJfijgrY2ZBh86lN4X1BA9MILEP6nT+PG2PX2qKmBpZ+XJwIusnY/Bxa/8vkbESrBUjh+Q+OCUVAAxk6zNnIJCiZHu/tukCPefTfeP/88XD59fYJv3uMh+pM/Ibr9dqKtWxHEvPVWzLMFC0SSVWoqbtu8efDATU7i88xMIQw5598MLMhlAc7xOyLc2iVLxEJj7AzmxOVjBLdZTE2FyyozU3yXni7cP0wwx5z93BiGmUeZOI6/HxtD0Fxl9UiIpLUiU7hwz8z2dszzujrcQOYX594e8jl4vrtcuCENDTERz1Oav4xwtPhwzDXjgtHeLhg75ayABEV9PQT85CRq4VJSoM1nZaF84qqrpuelnzghyMkefRSLAhd+pafD2t66Fd9/8AG04NWrkV3T1yeYK/v7A1MnU1MFuZlZARYjM1PwA7ndCKgGo5O2g9uNRYotEplFlK0MrhHggrLxceyzeDGsILcbv9vlwvhGRgQz6tVX4/vi4vDHmHAIN4fe64VvPzdXpHWfOoULziROvF1Pj8jvN1b33nprYNGYHFucAyjNX0Y4WryxMTSTPNmZccaMn2PH8GC9/HLCcfaYobISQr2tTWSsZGbCR80p0K+/jgSq11+HgJQ12HvugUB3uRAQvvFGWN9/9mf4KyoSLhoiLARbtog1Xdb8/X5Rm5OaKjRtmSEzMxPuJWbe5IwafkxCxebNsGQWLBC8PFyRLAep2RLh+AAT2nV0YMydnUgWGx9Hm4fUVHE93G6V1RMAJ/TpVqipQaXcrl3ot/3gg6i8v/NOaBzMSpieLrrqyM2arLJ45rjPr9L8ZUQadHGqWVi1bkxAzh4zNDfDpTMwAGFGBAE9MABBe+4chDlX9775JlxDDDtOGplDh/vlZmSIpihyv1zZr8+FU7qO2+F2B7p2mFF0cFAEj626iPGx8/JEOjgjJQVyZOFCJIscPSr6EOfmYjHo6MDYuR8vp4Hm5gqKCC5+6+5GptFnP4tK54ICldVjinBjbFbCe2QEx7vpJrGtGRWElUJZVYUbN4cJH0r4y3ASdOGovaYR7d4tbpjXC5/FwABendzMJK3m5WbteXkQ7pmZEH55ebCazaiZjb50JiYzorIS9AbXXAPLITcXFkVjo8gEMjJgMrjYyhh85fx7tu6Hh/FnxrnDwWe/X/y23FzhtsnPx3iGhyHgN2yAb57pKsrKhN+faZk597+3N7AzGS9iXV2ikLSsDI+lEvoSIplnVsLbjMufaLqiaKVQstI3hwkfSviHCo7aE8GXsHMn/HalpVC5OjrwUDi5mTU1kALnz8Nuj1Zu7yw3gg4VnMe/ZAl8+ePjEGpr1kCz3bkTmSyc4VJW5jxFk62Kzk4sMI2NIhjM+fGahsti1MqJRGCV6ZfZDVRais+YioHjB2bav98PuXLyZGBP3cJCfN7ZiQUhKwu/9ze/gb6wdy/Oz6mdbBVduhQo7IkC2zV6vXj/hS8kWStGp4gkh95KeF9xhTNl0a4uqLAQ7+dI6VPCPxSwds+E8gcPQj3jh6unB3l2PT3OtP+6OgR7OXrI1M6R5vbOIjlUOJDdNiMjEKL5+SiW5IDqxo1YR0+dwk9ZuBAunWACbflyhFFOnMCxuruFBs/uWKLpiwkHWllQp6eL7JviYtFMxu+Ha2XNGvjbOT2UM31SUjCXf/YzrL8ffCCYSa+8EsrexYt4hFpbcfzFi3Hcxx6Du6ulBd+73SJjSe4NTBRYCZySgmsnB3cVcZuESNy5M5GCGSMULkr4h4KaGqiW7Dc4fx4NmzMzIZm4cerYWKD/3koT370bkqCsbHplcLjae5wUj1m5bTgTqLsbApx5a0pKnGm099wDimdNg2J18aKomPX5QMF07hwoWYgEX45Mz+x2o7bg5EkI6PZ2IaDT05FX/8c/IvNmcBD7cJA4Px+3kQiPCOsD69dD8J8/D83f7xfunYsXYa2MjmLMV12F4jHOHuSAr8sVWEvAC0BuLgxHhiJuMyAcAT6T1vMcFHSZQWX7OAVr/T09UAmzsyHgz5+HpPJ6RZfxyUmob7W12Ncqqm9XU2C1j1musvzZHDSCjibYKmhrw2XMz0eCxdq1zpqLb96MwCcHW/PyEANYvRprckMDBPUXv4isIHbrsFafloYmMexvLyzEbf3oI9zSW26BG+bUKWjn3GDd78f/GzZg3f3+91F4lZIC4c69vktKYIGwNs+UMA0NcP2+9x7Rd74j4oh+v6hdkBu3swsoOxttGBctEtcgKYnbIsnhN0OkmTh245EzBOW/WS70UsLfKVjr5z5/RKJzSGoqnMILFkBlvPdeSKwdOwI18cpKFHNxG0i7FpFWaWlmDyV/VlU16+RQM4HNmyGw77sP/n8WbE412rIy/N11FzT98XGs0UwNfeoUlKxVqyBUOb8/Nxeet54erOdLl+JWZGZioSgowPp/8iQWpakpfJ+eDu0+LQ0CnHsADQ8LUrfRUcHzX1aGR2ZsTDRrn5gQhqPHg0Vn+XI8UhkZ+O3Z2TgHN3j7xjeInnoK+yU9cVs00ybDTQs1KmFzmMbpBEr4O0VdHWa21wtV7tw5kaPX0oLZPDAAbV+u3JM18eZmpKDU1Nj7/ay0d7OHUv7swAFInFluBD0TWL58Ou++U42W6wD6+hAr4Gbo7OdfswavJ05AiG/dCrcMZwPl5mL7+nrsf9VV2GdoCN9duoRxcM4/t4/0+bDtW29B4790SXT4TE1Fgxq3G8KcKZcHBrBtTg4E+9atWGRefx2PGZHg9OEG8fffjwXsX/8VCyQ3ZL94Ea9JF+yNJIffDE6tZ6N2b1TCojWeGYLy+TvF3r2BZplVi0e5vSJH9bnbT08P1LODB0X00ej3q62F2mmWlmaWq8xSLScHau3k5PRcyVn2JUYDnBFEFNh+8JvftN+PeYEGB7HW5ufjcqxejVf2s/f0CI193jzcStbks7MFNfTgILbhQjAusmI+Hu7hS4Rb1dqKcXKjFubmZ759Xcf3S5aA0qKyEscoLYXfftEibNffL1JLuW5geBhy5Pz5wOC3VfwkaRBqDr+dPz+UtFA5sWLnzkAlrKgIEfgY5u1Swj9cOPHPVVWhevdzn0POIUcXm5vR4/OHP8R28sN4+DAWB+71efQoJAIXhch8QH/3d3jwuMPJbbdZ9waNM4TTXJyDxQUF2I4XjGuvRaFYUZHYdngYl8jvR9ZefT0WGW4FefEiBPbIiODKWboUi8HSpXifkYHtmUp6chLnzM6G8B4dFbn+ug4BPzER+LuuugpxALnT1sCAcCv19ooCNWYqXbJEpXP+N8zaKQbLtLPLhpMtcnn+GQW40drgoi+XC3M9JwfbxXDtjhL+MwmPB6kdb7+NlBG3W3QQlx8I+WGUMwGam0UaKBeFyHxAFy7Ar7B6NT5LsK5foWq0TBvBfEDj46CK4EVgchLK2NgY1uDMTAj8U6fwP2fgcFeuzk6Rx3/ttRDA776L/3Ny0KT9wgUIdbdb9N/NyRHVwbxw8K1xuQJ/Fy9YXPTGY925EwvNmTMYHwd3S0shb/r6VDonEQUK69OnYXqlplrPgWDFmPL8a2kR889oPcvWRmcntP2tW7FfRgYWgHXrYnpOKp//TMHrxYNx//14GMrLEQi++26im28WDwRTxa5ciQVhzx7BE7R2LdEDD8DhvGqVeCg/+QQOZCKoqDHCEjjXaG6GACWCa+bdd0VbxG3b4E1rbsb7a66BYC8pgZZeUCCKq5ifa2wM2T1bt2LbtWvh4l23Du/vvZfo6afhd+/uhkDu7cV3S5cKt43bLSyD1lYIfAZbOBMTqBt44w24oW65BZbHDTfg9m/cCI2fUzpVOudlsLBmjqz0dFyYI0fMt+fEjY4OvMr+fK8XN+uJJzD/1qzB/F2zBvNS3u7gQcw3jvuxZd/ZCc3CLP4XY1Ca/0xB1gzsfPHvvYcFoLsbUsnY2Yf9mHIf0B/8AJLtU5/CPk6bTic4mDaioADaMvvb8/MhNJlaYdUqbPuNbyDo+/HHEM633gp/+vnzkCFr10Lg9vUFuliM/Pjr1kEWcHWurkPzz8zELZ83D9/t3IkFwUxjHx4GIRtr/x4PeMNOnMD3moa2lpz5lJTpnGZg96uRI6u8fPq2crr2/PmIyMvav2yBy7E0o9+eFxDmDeFYXm0tbgrfGDn+F4NQwn8mYAwaWfni29pgBRQXw7zcuVO0h7MKOg0OQnoY94lBn+JsQw4Sc8B0fFxQPa9eDcXuwAGxz333BbaN7OzE3NV1LBjsQrJzsVRWisIspoXgjlsbNohAMwdzjRq70V1lpLHm8XEjF6fB76SB0yBtTQ2EdVeXcJWy9i8HbJ97Di6kG280jyP88Y+wOIqKYG1s345FJ8aFvRHK7TMTcEoNXVEhcgH9fiwGExP43Gp/q33iMJ0z2mAXCgtPp9qyvF9bG7Tv664TsiSYi6W5GZ6B667Dmjw6ikWA4whjYwgqW41Bdlcx5HPK40vadE47OJ1vR45AYeJ0bb9fuIhkS7u+HotEWxsEf2troIuIeb7LygTF8xwUaUUKpfnPBJyUb3ODiMxMqKl+v9Ai3n0X1UZmaaBW+8RhOudMgIOpoWrLchCWXUeMYC4WdjctXgxytY4OEeNvakJcgVM8zcYgu6uszpn06Zx2cEqXsGkThDunXX3+87hJGzcGZgx1dWFu1dbCf+92i0XimmuIfvtbHO/kSVjucWp5K+E/E3CiAXCDiNJS8ZkZH7gMjwdaSCj7JCnCSRUlCq++wLgPN1D//OchD44fJ/rDHxDENdPYw61pSAo44dhxqnF7PIJre2gIwn3ZMvQOPXsWq29hIcw3buKwYAEq8jiOUFEhKvvT0mAdyLG6OIIS/qEgmmRPdXXQPl56CeWl6en43E6DjxFCqHhBONpyOIuGcZ+2NriAOHe/uFho9mbHCXehSgpEi6GWq/d27RI5/ByHe+ghpHU2N+M7zvXt7ha9NBcvxg1imtbUVFjfr70G7T8O56AS/qEgmlTJe/cS/frXeJjuugspZU72mW3EeG+AmUC4iwbv841viLo7RrC4gXLrmCCaDLV2HbXcbqR1DQzAT/jFL8Lnf/IkhH9jIxS05mZRrcc3dHAQ38WZv59IBXydI9r8IW1tRH/zN4gGPvWUs+NFm7nQCeKAoCrWEAkvUUIi3Oc2mgy1stUs18QcOiRo2sfHhTvo1Cn8PzCA4ouGBlgCra1I2+7pEXG3GObvsYMS/k4RzQfR6yX68pfxIHE5eHV18P2qqlBV5GTbaCDaC16SQCaWS2qmTUY4CoRZ+mZlJWpdwnkOzWiU9+2D1s807WvWIDWstBTjPXsWLVuvuQYZPT/8IYov7r0X8brly8H/XVgYl8qREv5OYEe/HA5eeAHaBTOBDgwQ7d9vfzyvF4J/YsK5pRAp4rw3wFxBpWZKCFeB4GePCKlTmgYN3eOJ3nNopGnnIkxO6zSO/f33A6uJ/X689/niUjmKyOevaVohET1HRCuI6AIR3a/rep/Jdn4i+ujy22Zd1++M5Lyzjmi2XfN6if7xH6ESchL45CT8i9XV1r7/qipYCAsX4vWFF1CRxL54q8by4UIuYS8ujmmCqliE8uFfRqiMmwx209TWIjd/YgJpmUVF05/DcONSMk271ytygpnLx1jly1X2cjUxQ+7cFyeIVPP/HhH9Qdf1tUT0h8vvzTCq63rZ5b/4EvxE1v7CcPg6qqshvLn9E7eMGhyEQDcDa/0ZGYIQ/he/ENz/RHg9eJDol7+MjlvIWMIex70BFOYIkVjMe/cS/Z//g5zZL31J8GPdeqvzrndOzvHhh3DvsIvn+uvBtbV7t/XYoykP5hCRZvvcRUQ7L///70T0OhH9VYTHjD1EK5Lv9YJbgMlemPIxMxMPN6d7GiFr/UQQxB99BP8k00EwZ0l/P1xId9wRmYZ+5AjS39LTYeKmpOD/OExpU5gjRGoxV1QgMYIZbM1okgcHIax37IjMMmWCRT4OUzSbjT0OM3vMEKnmv0jX9fbL/3cQ0SKL7TI0TTuqadp7mqbdbXUwTdMeurzd0e7u7giHFoOoqUH1YHExzFdm/crPR7ApK8tcK/J4YIoODUG4NzUJVxHTQXA2gtstXEiRoLwcpeu7dsV1CbvCHCISDbmtTXBYffgh4mNnz6IxA+feM93JxYv4fHg49KAyZyHxQtPdLShWE0C7t0NQzV/TtN8T0WKTr34gv9F1Xdc0Tbc4TKmu662apq0iosOapn2k6/q0Nli6ru8nov1ERNu2bbM6Vvyiro5oyxb839ICzT87GxHBz3zG2m945ZWCH3hkBFZAXh4CTVlZRL/5jSCsZ4vCSvt34h8NpZuRgoIVIlEUZA6r5mbR2ebiRTzjy5ZB4L/0ErZpbETmTbDnVH7+2V20cOF0ssShoYRoimSHoJq/ruu36Lq+0eTvEBF1appWTER0+bXL4hitl1/PE1xDV0ftF8QT5HSznTtByVxWhsCRnWYh77d2LayGTZvg9qmvhzXQ3w+tf2wMWpKZ9u/1Ej38cGCswAxOibIUFGYCXi/Rq68KDquiIjzT2dnIvFmzBpz7TNs6PIz/P/oouPZv1mf3n/4Jx0kyssRIff4eIvo6Ef3k8ush4waaphUQ0Yiu6+Oapi0gohuI6KcRnjf+EY5WZJwUfj94hIeG8P/kpEg/8/vhLpKzh6qq8EBv3WqvISkaCYW5RE0NlBs5m+att/DKPDtVVaBW4Iy5tDS4bFparJ9TOXWT++wWFmKfrKykI0uMVPj/hIh+o2naN4moiYjuJyLSNG0bEX1b1/VvEdGVRPRLTdOmCJbGT3Rd/yTC8yYfvF6i73wH3MBr1ojPz55FQ5jMTGhHfj+0mO3bUXYu7//00+hA3tIimleYPdzBFqYkpHxQmEUYlQ+fD8+s7IY8cABdeaam4AJKS0PCQ2ZmYNctGWzRulwigMypzC4XmPhcrqQhS4xI+Ou63kNEN5t8fpSIvnX5/3eIaFMk51EgPLjvvgvSGLkjWHs7HuK77hKfmT28csbQ0BC0pXD9+E45jtQiEX+IhXtmVD6MefVuNxInWlvBn11cDMUnNRW1L2ZKjRzHunBBVNbPmyeKLZnlkyjhtX4iRewWH+AH9wtfmB6IeuwxonPngvcO4DqBtDTs29gYHhVtKGRb0STCU5gdxOI9M3NDbtkCUsSMDCg0LLSJzAW3HMfq6MBnFy6ApqGsDO/jrBNXpFDCPx5gVyXptHdAVxdy9fv78dnQEKyGUDUcpxWb0WRkVJgdRPOeRdOCMHvGvV5wZmRnC4VI18U5jZAXkNJSZAz19WHReOaZyMYXp1DcPrEOq7TL9nZrpkQjiyKnmJaVib8dO2DuWvlHQxmL2RgUL1D8IZr3bCbZYDn+NTwcOFa7c3LG3L590PBXrCB64AHr2pokgBL+sQ6rtMuKCusH3TgJzBgNH3gAx+JtnNDuOk0BjTYRnsLMI5r3bKbZYKur8cyNjYmxVlYS/e53wc9ZU4MYQlNT0ismSvjPNYIJXbMqyYkJoldeMX/QuUx95UrrSWA2OZ1oajyWhgbs19BgXpug6gTiD9G8ZzNp9TFFCseteMzNzfizOyeTFY6PIzOIO3QlqWKihP9cI5jQtdLauRm18UE3lqlbWQby5JQLXuwmAo/lgQeQYWFG+eD1YgwTEwldGp9wiBZZWagWRKiNXjh+lZ4uMnQaGpAKOj5uf04mK0xJER26klgxUQHfuUQ4ATY76oXBwell6mb0t8b9ueCluDg47W6wMdfUII3uwQdjJ1tEITiileUSKpkbKz+lpXDFOKEdue22wD68N90EYd7ZKc5tPCfvyz0CJidnlrAwFlJmg0Bp/nOJcMxju8kl86FYlakb9yeClif7T4P5TK3GrDp/KYRiQRgrbsOlHXnxRSRAnDoFS8DsnLzvzTcT3X03unHNJGFhHLQ/VZr/XCFc8jQr6oXaWpS7y9QPZmXqxv2bm7Ftby/eW2lNTz5J9NWv2o853MYdComDUISoXHHb2Ei0YUN4tCMrViBrp6zMmpBtNilL4iTNWQn/uYJT89hoPlpNLo8HFY+LFxMdPQqh394+vdLXuD8XiREFTgx5UrAWMzhoPeadOxUTqEIg7FwfxorbjAy8Njdbd7Sze/YPHrRXOmazeCtOlCAl/OcKTjURpxWXfLwjR+DvZ86TYJqNEx4f1mJefRWvZmPmlnfRaHWpkBiwe3blHr1nzoBzqrUV7sennnLejCjW6MdjbTw2UMJ/ruBEE7EyH40aldcLYfvjHxM9+qi9+RsqZC1m40ZrwqvHHhNpoKdPg1ROdf6Kf4QbuAzm+pCVlYEBPGODg3BbNjba97OWEc3+2tFArI3HBirgG8uwCq5yYOzhhwNz9DnFMpQAsl2qXShpe2ZpoPv24eGPVpqfwuwj3MBlTQ0qcOvrzTn2+XnhvhYLF6Kp0YIFsFoPTWOHN0es9dONtfHYQGn+sQorwbttGz53uTChXniB6J13EPR64QWYy/L2wcxNJ6a5lRZjZoHI2t7IiL3LKhZJxBINkaQchhu45P3GxpC+mZdnvf/evYKnp7w8MIVzaMjc0jXuH0uItfHYQGn+sQo7WofhYbSzy84m+sUv8P7SJaR5trUFbm+nsQVLzQymxRi1QtlSGR5G+p7Zsb1euImclOMrOIeZJRVJyqHxfrKlaXUueb/hYbhv5s/Hq12HrWDVxcF+g7Igw4IS/rEIqypZpnUYG4NpnJNDdP48HvrOTvhLjx2D392JuRmszsCsuphzoo0LR3t7oKUyNmZdQan4VWYGRiEZSd2F0fIcGyM6fFi0BrUTyHV1eB4GBtBPemBAMMianceuItzJb4iDnPpYhHL7xCKsqmQ9HjRrP3lStJ1LSSE6cQLBsVC6EAXLSgjmLjCms/EEZrOd+wUYG2vrujW/SgxmRMQNzFw0kaQcyto438/cXGTifPrT9u6g3bthmRrdOGYMssEqwj0eUQtw7Nj0QHCc5NTHIpTmH2uw03RkjWpoCK4eTcPEsqpslI8rm8aRmNpmC8errwrtrbYWY0xJgUUiH1vxq8wMrPiawmXplF1+fD/T0sCrEyyxIFT2Vyut3lgL0N+PxUferqoKi4J6hkKG0vxjDXbaGgeTuChLRrAuRMbgql2dARdsmWlTXi98v34/snqIMPHkNFC5cIxInKO2VhDOEU3nV6mtFQvXnj3Og4sxzqESMYL9xmB8TUShpxzys8TB2B07cIzBweCJBaHUsNhZJsZagKKiwDRQ7lDX3z/dwkzUZyGKUMI/luCkQCScbAIz09juOGxqm03Kmho0jDf2EiYSkztYJebNUttn2U3l8WDhIEKtghMhlQwZQ8F+o5mm3dWFxdXqHoVybvnY7e0isWD9evNFJZQaFrtn3VgLkJ0t0kDvv1/0pS4qwuKwcmXM5tTHIpTwjyXMVIFIMA1L1ix13XpS8ndmvYSdIJi1wbEAIvwfTINLBn+vk99o1eM2Gj1pjcc+dUq8yuSAoSwqXi/Rn/4pjnHbbfjMahGxSwP1eLAYjI7iPTdgV4WFjqCEfyxhJsinnGhYsmZpR9PA35ktIk7cL0ZBJO9z+HBgLKCpKfiiNxscKnPtVnLyG1lIzsQ4Q108vF6kH2saAr9WyQKvv47fxQKbwYoA/xYrhaiqCtdk167ARSEaVe1JAiX8YwkzUSDipFBL1iyLiqxZQ7u77Rk9Q3W/yFzur7yC7B+euD09Qvvnxtxf/SrRs88Gt1CiOfnn0q0UCk+M2TjnYuHiNF4iWB/Ga8bdtBYswPvSUqJ//ufA8Xk84rfU1Qna5iuuQGyICK4ftzsuaBRiFSrbJ9HhpFBLztzYscM8t7+83HoRCSef3Mjl3tiIz9PS8Cdr/zJ9BWcgycHAt9/G9jPRMjCc3xWtgqNIs2aimf/u5HfJbRLHx/G/VTet1FRYec3NgeMz/pY9e0AZsmSJ4N5/5hlhpcYBjUKsQmn+iQ47ayKUXH87l5SdO8gK8qLT1SVSQ43Cgi0Opq+49VZhoUxMoO5hdNQ5i2kocOpWkq+TUQOPRPuOJGvGLmMrHDixLGTBrutCsMvbHzwYaOFduhQY3zH+lqoquASNvyOOaBRiFUr4JzOCuYTkCW812TggF4r7xbjo3Habtb+WM4QGB5Fl0t2N4rEdO9DH+I9/hFWyaJG9vzdUIRyOy6W0VAiqykoI77Vrw3cbRZI1MzISvXiIVdBZfj44YB9MsDc3w0qTs5B4kTDrCRFKi1GFkKDcPsmMI0eIPvpI0EGEWlZP5Nw1Ec4+PIaCAqTyLViA18JCCNf9+yFUWloEd4yVeyJUF0goYzx4EJbL/v0YR04OBFplJfLQnbqNeOzt7c5dR2buL+ZVCrfAy+occlGX8fmorg4U7CzcZbdOXR0W7+ZmourA4QAAD5VJREFU/F28iDH19OCZM15zotBajCqEBKX5JzPKyzEhzUrr7XL9ZRw5AtreiQkRjCOyd7+E6s4YHIRbJyMDgqCtDQLyzBmiVasg7MbGrJlEw0kJ5TE2NCAlkYONZmNsbsaYRkawUDF1RV8fzr11qzOtlReowUGc34m1wOOsrRVNfHQd545GMNSpZfHii3j1evHH0LTA+o8tW4j+7d+IHnpo+li4JwQ/F05ajCqEDSX8kxV2AjEUl4fdAmKFYHEIY5zh1KlAofLxx3Ap+P3QMLOyEDBOTYWmfdVVkfcW5jF6PBBWZr9P9mGnpmIBOnsWAm9qChpqTo6z6lO+5hzbuOWWwPoKO0pjdr1dfTXOWVQErToaKcNmFhBbFlu34rPFi3Fd33orsrqMcFqMKoQNJfyTFXYC0WmxWbSKrOQG8d/7HgLAdnEGj4fob/8WQrG/H58NDYHwLjWV6Nprpwc+w0kJDfb7ZB82F6e1toJpNS8P45uaclZ9Kls5Ph8Wt9RUUGncfruzCl++lzt2EP3VX03vtfDkk4IjX77udnEQMyutrc3csqiqQoaWUzLAYIuwCurOKJTPPxkh+9Lffhs+dNmX6rQbkZkvOBywu+Mf/xH/u932vt26OrgPysrEH3PPcMUoC/nq6tBjEk5/35EjiDdMThLl5xOVlMAaIQIDptuNRWBgAG4ZO9I9s9jG0BDRH/4Ay4ODyI89Nr03gtniVl09vdeCMebhJA5iRuv92c8SbdiA88j04YcOhUYGqPz3cwql+ScjZC2zqQnCJiNDaGLR4mZxAtnd8V//BUHb0iLcNWaaodn4OCvIKORffBGuoVBdIG1tRD/6EVoMWv2+8nJo+qWleD8+DkGdnw/NOCUF7QmXLbOnWpDvh9+PazAygmA8ESya8nJYGSdOBPIeWbll9u+HBXHwoOj+JlswXCQXjtW2d+90dxi7noqLra2kOOltmyxQwj+ZwGb+4CAm3rFjCGIeO0a0fXtovtRoTWY+Tk8PxpWbCwHIAVynQokzl4yB5xUrcMxQ8+wrKrAA2BGYGV0izc0Q/EuWEHV0QDs2S0E1ulvk2MbQEP44iJ2SAvfPmTO4RlNTgemTZm6Z9nbEGdLSwIMvUzB3dgpXUripoOH0DpgJ6hKFiKCEfzKBzfyHHoIm6XJBa3XaAEZGNCYzB0zPnkWA0u2GD7+0VDSDMRNKZr5qq8Aza6ih5Nl7vehPkJsbSDnNv4+5Z2TaaZn6mLnnrSwYY8GUGefRI49A2J87h4D2hx/iNTU1kPfIbN/vfhf7ZmZC2P/2tyDjI8KCUluL6719Oz4L1WqzKsSKNhutwowiIuGvadouInqUiK4konJd149abHcbEf0TEaUS0a90Xf9JJOdVCAOytlZZiSBlJC6baExmDpieO4cqXbcbwqmjA0KOW/+ZNZY3VtHKv+34cQjmwUG4bnbsCF24bdpkvTDK3DNG9wsRtPTcXGjwRgvGSZCcj9XbiwV2YAD58ZOTiAv4fNb7VlVhbJmZuJ4dHSI9dsUKQaPBriSi0Ky2megdoDAniDTge4KI7iGiN6020DQtlYgqiOh2ItpARF/WNG1DhOdVCBWyttbcDKEWThA0muCAqc8HV8+8eQg+5+fD175zp7lmayw+M/42jwfbfPWr8MlzAxmrQKRcWGXsRWwMTFoVvxm55/lzblUpB16DBcn5WMuXI211ZARxmVWriO6+m+imm6z3razEbxgchPUxPIy/jz8WY/P5sNhy97dQeHHM3H1dXTin4tmJK0Sk+eu6fpKISNM0u83KieisruvnL2/7ayK6i4g+ieTcCiHAqK1NTMAd0NDgvDBrJiAHTM00bBbMsnuH2/Z97nPTXQ5cXDU1RbRvH1wlGRnwpd90k3UgUi6s4ngIETKhtm+fTnlh5tvmRcrYxYxICEKnaafygufxEB09iuY5y5YFFp0Z75fXC+G+YgV+9+c/L/o633YbrpORFz8aPRmi0TtgrqmzkxCz4fMvIaIW6f1FItoxC+dVYBi1tZtvDs/Pb4eZ4M4xc+8Y2/bJLofTp8Xxjx+HNTE0hMybtrbpMQSzwqpXXkGA9513BJ3wypXOhbeTDmlOg+Q8Prl5zuHD1kVnNTXQwtPTRXopc+WHQoHMnPxE09tpyr9Prs949tnAGoJQYUYcpzCjCOr20TTt95qmnTD5uyvag9E07SFN045qmna0u7s72odPXjjN248E0ebOMXOvVFVBq/b54LPWNOFyaGiAReD3Q/CNjWE7Igj+2locX/7NPIZLl0Rh1aZNRHfeiYVi/ny8PvEEhF44PEYyQr0PRhcRN2U34wri4Hl+Pvrr7tpFtGYNxh4qBXJNDRYqjyc4R5ORajschEOdrRAxgmr+uq7fEuE5WolIatVDSy9/Znau/US0n4ho27ZteoTnVWDMdKZFJNw57D7w+eDOWLoUml91daB7hwXS0BB84OPjEOjsctiyBS6OxYtB85yVBYGZloZts7ICtWW5sOqNN0Rh1c6dyJFvbkaOvtwwPNIMp1Dug1mdwYEDEO7nzyOF1FiV3dwMN9qZM1gA5W2cnlvm5CeybqdpR0cRqvYfDv2GQsSYDbfP+0S0VtO0lQSh/yAR/c9ZOK/CbOH/t3cuMVJUURj+/8A4GBUYhTA+iA/ABxsfmSCPjVETHwvGF4lu1ASjLty4M1E3JsboxsSEmBgcHxvFZ8CAEp/RjTAkOiqOKCLiwCAIhsRkMjLjcXGqpmtqurqrp6urpqn/SypdXX1n6tx7u0/de+495zQTOyck3JK5YoUrlr6+yead997zHUDd3b6TZWRkchaoMCjYzp0uQxgJMtwrv2+fj07ju3OijlWjo24OGxz0B0lnp4/8X37ZR9N5bleM+xkA3hYLFvgMZd68yco2XDzv6HCTVRhfKJ4WsZ5iDh8i9dJpVgtHkbQ1txZZOQuKhmlqtw/J20kOAVgFYCvJ7cH180huAwAzGwPwCIDtAAYBvGVmu5sTW8wYsnDbj88c3n3XR9cLF/oolnSFNDDg552dkxUT4Plily8HVq3ygGOLF7uz1Vln+eg+au8GpjpWDQ25HLt2+XsLJp5z51ZG/3GZs8rYVa09Qj+D/n5/8G3a5Mp2/343R/32WyWMNeAPzTVr3GR15pm+UL10qdvs05rk4slWzjijkk6zWkiJaqG2G+37Zk1pYto0pfzN7H0zu8DMOs1skZndFFw/ZGa3RsptM7NLzWyJmT3drNBiBpHFjzdu237lFR+Nj4xUFi6PHnWTxtiYzwjGxnyku2NH5X989ZXv6b/uOn/t6PB4O93d/vrhhxXFFMas+eYbdzILj9Wr3dzT1eX3+ecfH0Fv3jxV5rR27kYfFKGfwbp1PrsZGfEZzNy5vh02bJfQDyL6AN6/30fgYY6DrVsr+QaqpVWM3/fAAT+vlk4z3l/Dw95Pc+Z4Gx061Hjf57EeJaoiD1/RHM3aweMzh64uX8Tt7a2YYo4fd6/kjz6qxNEBXClde+3UmUO4Kyb0YI6Wr2eWuOIK/7s4S5ZMlTm8X09PJbF8UjTLtDtZou0xOuptceyY2+9HR6e2y6OPel2THMz6+iqOcx0dtes/MOD/MwwxESXan9XCUQD+/sorG9syLM/fwpDyF83R7I83PnMYHnYbcjymTq0AbdVyCE/3oZSmPvE1jg0bkpOvNLoYHm2PPXt89A74fcjqsYbiDmZhZNEjR7z8+Lg/TEITTpIMTz6Zrv5S2KcEUv6iWOJK+qefKq9RG/1ll1VXOkk5hBt1XkpLtZlKNLF8UjTLtIvh0Qxi/f1uepo920fvs2b5tXisoSQHswMHKg+PMHRGmtmPKAVS/qJYsp45tDquTNJMJUwsX82JrJGdLNEMYocPe7n58/3aiRO+qJqUNS3elk88MTlg3vh4ZZ1Eyr/0SPmL9mY65p1mQglE73fypPsinH66K+qVK6unj5zOgymt/b0W8XwDQGWdRJQeKX/R3kxn5tBMKIF43J34onJUuTezGJ7W/l4LxdAXNZDyF+Uiq7zDQH3lWvTCaNH3FzMaKX9RLrIMJSDlKtoYJXAX5UFJxIWYQMpflAeFEhBiAil/UR4USkCICWTzF+VBNnohJtDIXwghSoiUvxBClBApfyGEKCFS/kIIUUKk/IUQooRI+QshRAmhhblKZxgkjwL4vWg5MmABgL+KFiJnVOdyoDrPTC40s4X1Cs1Y5X+qQHKXmfUULUeeqM7lQHVub2T2EUKIEiLlL4QQJUTKv/W8VLQABaA6lwPVuY2RzV8IIUqIRv5CCFFCpPwzhuQ6krtJ/kcycVcAyZtJ7iG5l+RjecqYNSTPJvkxyV+C166EcuMkvw2OLXnLmQX1+o1kJ8lNwec7SF6Uv5TZkqLO95M8GunbB4qQM0tI9pE8QvKHhM9J8oWgTb4jeU3eMjaLlH/2/ADgDgBfJhUgOQvABgC3AFgO4B6Sy/MRryU8BuBTM1sG4NPgfTVGzOyq4Fibn3jZkLLf1gP428yWAngewLP5SpktDXxXN0X6dmOuQraGVwHcXOPzWwAsC44HAbyYg0yZIuWfMWY2aGZ76hRbAWCvme0zs38BvAmgt/XStYxeAK8F568BuK1AWVpJmn6LtsU7AG4gyRxlzJpT7buaCjP7EsDxGkV6AbxuztcA5pM8Nx/pskHKvxjOB/BH5P1QcK1dWWRmw8H5YQCLEsrNIbmL5Nck2/EBkabfJsqY2RiAEwDOyUW61pD2u3pnYP54h+TifEQrlLb/DSuT1zQg+QmA7iofPW5mm/OWJw9q1Tn6xsyMZNIWsgvN7CDJSwB8RvJ7M/s1a1lF7nwA4A0zGyX5EHzmc33BMok6SPlPAzO7scl/cRBAdHR0QXBtxlKrziT/JHmumQ0HU98jCf/jYPC6j+QXAK4G0E7KP02/hWWGSM4GMA/AsXzEawl162xm0fptBPBcDnIVTdv9huPI7FMM/QCWkbyY5GkA7gbQlrtfArYAuC84vw/AlNkPyS6SncH5AgBrAPyYm4TZkKbfom1xF4DPrL2daerWOWbrXgtgMEf5imILgHuDXT8rAZyImD7bAzPTkeEB4Ha4/W8UwJ8AtgfXzwOwLVLuVgA/w0e+jxctd5N1Pge+y+cXAJ8AODu43gNgY3C+GsD3AAaC1/VFyz3Nuk7pNwBPAVgbnM8B8DaAvQB2ArikaJlzqPMzAHYHffs5gMuLljmDOr8BYBjAyeD3vB7AwwAeDj4nfBfUr8H3uadomRs95OErhBAlRGYfIYQoIVL+QghRQqT8hRCihEj5CyFECZHyF0KIEiLlL4QQJUTKXwghSoiUvxBClJD/AZV7k+ol5pQiAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff3a503a2d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(X[y==0, 0], X[y==0, 1],\n",
    "           color='red', marker='^', alpha=.5)\n",
    "plt.scatter(X[y==1, 0], X[y==1, 1],\n",
    "           color='blue', marker='o', alpha=.5)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "同样这也是一个二分类问题。我们分别用标准PCA和RBF核PCA处理数据集，然后比较他们的结果："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [],
   "source": [
    "scikit_pca = PCA(n_components=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_spca = scikit_pca.fit_transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcUAAADTCAYAAADwOQi2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJztvXl4XNWV7v1uqaTSLEu2bHmUjQ3GA7YBYzMkFzM5QIchZrBzOySdTi5Jup3vI908N2mI+6PbJND5kjQkOCQkGDLcZgi2sRIJCIkFGAfkASzjQbZla54sayzNNez7x6vd+6hUGmwNVZLX73nqKdWpU+ecOke137PWXoPSWkMQBEEQBCAq3AcgCIIgCJGCiKIgCIIg9CCiKAiCIAg9iCgKgiAIQg8iioIgCILQg4iiIAiCIPQgoigIgiAIPYgoCoIgCEIPIoqCIAiC0IMr3Acw0kyZMkXPnTs33IchCMPiwIEDZ7XWGeE+DoP8roTxzlB/UxNOFOfOnYv9+/eH+zAEYVgopUrDfQxO5HcljHeG+psS96kgCIIg9CCiKAiCIAg9iCgKgiAIQg8iisLI4PEAP/gB0Noa7iMRBEE4b0QUhZEhLw/YvZvPgiAI4xQRRWH4eDxATg5wySXAH/8o1qIgCOMWEUVh+OTlAd3dQHIyn8VaFARhnCKiKAyMxwNs3sxHKAvQWImZmXydmSnWoiAI4xYRRWFg8vKA7Gw+QlmAxkp0u/na7RZrURCEcYuI4oXGQFGiwe95PMCOHUBXFx87dvT9XEEBoDVQUmIfWgMHD47yFxEEQRh5JlyZN2EQTJTookXAHXf0fS8vDzh6FHjmGf5dVgZERQFKAaWlXOb83KZNQ9uvxwM8+yzwD/8AJCWN3PeJlP0JgjAhEEvxQsLM/82dCzz+OFBT0/c9txvYtQvYto2WYX09RcXtBo4dA159Faiu7mttDmaBfvObVnTHAkkREQThPBBRvJDIyQEOHKCoVVUBW7bY9/LygLY2oLwcSEkBnnoKKC7mey4Xxa61le7SLVv6Cs5AIpSby+Vud+8gnNFK+JcUEUEQzhMRxQuFqirgX/+Vlt/HHwPTpwOvvUZr0YhIZyfg9wOTJtFtWlJCQSkrs27UigqKnFNwBhIhjwfYupXWZnk5hdcIpxHSnJyRFUdJEREE4TyROcWJhsdDKw8AvvpV4Le/5bzaj38MVFYCaWlARwcwYwbQ0ECr76qrKFbFxRQvlwuYOZPCuW0b3ak7dgBZWRQxpSg4DQ0UHK2tCDU0UOROnOB6s2dTXDMyuI/OTmD7dmDvXh7PJZdQNKOiQs9zns/3D5UicsMNMrcoCMKgiChONHJzgeeeowh1ddHdmZUFvP46xa6uDoiPp8WXlAS8+SbX+/BDIBCgpQhQpM6cocDs2kVx6eqipen3A+++C1xxBQXO66XATZ/O9bZu5X6U4jZjY7nvxEQKb3IyxXXuXGDaNIrmokUjI14DpYgMV3AFQZjwhNV9qpTaqpQ6o5Q63M/7Sin1E6VUkVLqkFLqirE+xnGFx0NBbGoCzp4Ffv97WoSPPkprbsYMICGBIpWaSpFYupSvo6OBefOAJUsoYEuWAMuXAzt3WpEpKaHQtbYCx4/TJVtcDPz5z3SNmjnIU6doMZ49S7crwGNqbQUaGxmwU1ND0T15kvsPdq2eL5IiIgjCMAi3pfgigGcA/Kaf928DcHHPYzWAZ3ueL1wGSjXIzQUKC4G4OIpPYyNForycIpiYSMutqYnitmcP0NICfPQRLbasLOAznwF+9ztgwwaK5ubNFLmSEuDIEVqYXi+tzQMH6A5taABiYvi6qqr33GBsLJCeTsFbuJBiePgwLdGWFju/aVyrw7UWh5oiIgiCEIKwWopa6/cANAywyl0AfqPJhwAmKaWmj83RRQjBEZr9RXkaK7Gjg1ZdZyf/PniQVmBzM5e7XLQWXS5g8mS6Of1+itXp08ALL/QOmNm0CXjxRT4efxyYPx9YtYqivGoVt3HJJcDUqcDll3O7mZnAggV8TJvGZRkZwJ13AnPmUJDdblqSZ87w2LSmuLa1ARs39g26kdZUgiCMAZEefToTQLnjdUXPsl4opR5USu1XSu2vq6sbs4MbE5wiOFCUp7ESXS7O/WnNR1cXrbjYWIpkWhrn8qZMAYqKKEilpYw4LS2lVRiqVJuzuk19Pd/3eCikyckU1vJyukYDAR6Hy8W/Cwu5za1buX5mJt2zSUl8TJ0KrFhBwayuBj74oK/oB58HEUhBEEaBSBfFIaG1fk5rvVJrvTIjIyPchzM0hjKwGyFqbuZzbm7/qQbZ2RQ9n4/rBwJ8ALTMZsygZXb77cDdd1MYT53ie93ddKm2tNA1Wlzct7B3cHWbEyesG9TjobiVlPAYamuBTz7h+nV1XNbSQmv0+HE+19fTekxOBq69lpboT39K6/Kzn+2bz+i8GcjNlcR8QRBGhXDPKQ5GJYDZjtezepaNfwYqt+Zcp6yMIhIVRdfm5ZfzveBUg0svpUXo9QL79tGiM2KVkcF5xooKulOvu46C5vfTzepycT9xcfxMZSXn/4zwrllDUeru5r59Plpz7e0UyLNnrVt20SIeW0sL8IUvMHI1PZ2W4uHDFMmEBAb4LFxI63TVKvt9g1M7Sks512mW19bS4ly6VFItBEEYcSJdFLMBbFRKvQwG2DRrravDfEzDJ9jyCTWwGyuxvp7zdqWlFJ4regJwg1MNHnqIAThZWXSVlpZSWPx+uizb2hgcc+QILci6OgotQLFpa+PfqakU15ISvj540OYh3nSTPb6dOyl2mZm0/hYu5OdmzeK209M5H3n55QykAShora3c75//zGNLSOA+jPCmpTEAaNkyil8gwDlRczPQ2UlL9sorub2NG4EnnrD5mCKQgiAMg7CKolLqJQBrAExRSlUA+P8AxACA1vrnAHIB3A6gCEA7gC+H50hHmGCLKFQOnbESAQpgRwetsvx8JsQbDh7kZ43lWVBAcZozh4/ycgqcy0XXaVwcRWXKFM7fxcRQgI0bdd48CpQzinPzZpvmYFi8mBbinXf2jlbNzqaYt7RQtI4doxgDFOspU2hhut0U0s2b+V52Nvff0sLjq6mhAKan07V7xRW0fouLecy5uXQBf/wxCxMcPmwLmYswCoJwnoRVFLXWnx/kfQ3gH8focMaGoVRc8XiAp5+mtRcfT1EwyfjXXmuFJHibl1xCS+xHP+K2qqqAW2+luzMxkQEvN95IAZk9mxZbfDzdpbGxjCxdsAD41rd6bz9UmsPLL7Ns3C9/yXzGP/4RWLnSWnvvvkvR83iAJ5/k/h9+mMf1zjuMSn3tNeAf/5HnoKCAonjgAJ/Ly7kdn4+WYn4+hbm5mRZmSwu3nZAAvPQSjyEvj2J5222901akY4YgCENkQgTajCuG0pQ3L485hgCtr7vvZvBJSgpw2WW9t+fx0IXY1tY3AGfLFgbEdHdTSJqbKZQLF1IM77uPc3NuNwXy+uuHVivU4+H8ZnMzLUHzHbZs4XN1tXWNer1cbgqOHz7M9T0e+xmAwrt+PecX09MpovHxwOc/z+NcsIBzoatX01JMSKCbtr2dj8JCCt7WrX0DcXJyeLy5ucO7doIgTHhEFMeawSquVFUB3/seRaO6mvNrJSW0lGpq6GY0eDzA177GYJbOTi4zlufJk8Arr9DSAuh+bWxkEE55OV2bAEXNBNycONE36jQUOTmMXNWa2z15kp976y0K34ED3GZTE4XtzTftfk3KTFcXj+3NN3sXFe/spNjGxNAaPHoU2L+f2121ygrn1Km0dk0OZFMTbxpOnaL1auZrq6spiN3dwPPPSxqHIAgDEumBNhMH48L71rd6u0mD3Xxf+ALFIz6e83txcQwkeewxBpuYFk5JSRSRt9/musXFFAJjtT36KIN0YmLofnS5bI7itGlcvncvxdEI5/HjtBwHqhVqrESvl9ZlTAwr4lx8MT87cyatvKws+5nSUr63dy/3n57OfXZ00Go1RcVNUXK/n6/b2+lqDQT43fPzKaqmm0cgQKGPieH3a27m+zU1zHvs7uZ8Y3Exg5U++oi1Wr/4xdG91oIgjFtEFMeKUCkYubkUmLlzgfvv5xzb3r20eJqaaA2dPs0gltmzGcVpAnPWrOFnTS3S1lbgvfcoNPPnc1+dnRQXn8+6UJOSgKuvti5Tk4Tv9/OzO3cyiCY/n1Zk8DxcTg5FRmu+1ppCtGcPRfyDD2xnDIDiWVhIgTtzhvtsauJ7ra205JxWcnMzRbO5mcfT2cmIWJeL5yIqimIPUBA7OrjthAR+3gQN/fWvXGfbNgp1ezu39Z3vAGvX2jldQRAEByKKY0GoFAytaSWWlgI/+xnw6U8DP/kJrbqmJlpbHg/djPn5tPIA696sr6erMCHBpky0tfH9Sy7h+/X1rFQDWLF56CGuv3kzxcaZZrF7N12WGzZwnV/+sm8eZXY2LbT0dLustZWpGC++2Pe7Z2dzOzExDIYJZv783oE8hYV0mUZH8/hqaymIZvnixbRCjSVaUMDnxYvpRm5vp3AeO2aPzZSRi4mhMD/5pG2vZa6PBOIIggARxdEheJDNzeU829q11tI7e5ZWodcLHDrEgbq8p6Kd30+Rq6qyCfMFBRzcr7mG4vf00xTMs2c52Pt8XD5zJufpurspEMFzaMYFGRvLlI+TJ23Sf3k55+e2b6e1FSqPctEifjaY+fNDn4dQUbH9sWkTRXTzZoqsyc1MSuIxp6RQDH/yk77byc6mlZqUZOc6u7r4Pc6c4XfUmoL50ktMDfnKV5jfaPpEjkQ/R0EQxjUiiiONiQY9c4aD7Jo1LNRdXEwBWrCAovPhh7Z4d1MT8JvfsBRbWxvFzO+nIHR320hLj8cm3FdVcb3ubj605vqlpUyNWLcu9ABv8ghvuYXrmvWysyl2zkbCc+b0zaM8ly4UQ8nHDD53pr5qbS3nBpViykh8vK21Grwdjwd49VW6Rxcu5M2DcbW63bxh8Pv5/RISeN527OD6Bw7w3C1ZIhVyBEGQ6NMRJyeHg7bbTfH76lfp9gsEGEX54YcMaDl5kut7vTb/rrWV7s4ZM+g6jI5m0vrXvkYxnTGDonXttSzrBtACMlVfTMBJenroCNL+ciSrq+1yU/D77FkK2lCiUUPR374G2o6zvmp9vT03xuoz85fB28nLo7Xt8dClXFfHz7oc93xRURTE+fOtG/X3v+fy4uLQqTHme0jxcUG4YBBRHElMZGZKim26+6c/UfBcLlo/BQW27ihgi3Zrzb9XrODDRIneeKMNWomOtpGccXEUv7g4blspWjgmFzHUAN9fjqTJL3Q2EgYGFovg7x0sHEPJxwzehqmvatzHAMUNoHjNmcPgo+Dt7N1La9Lvp5jX1/fuEuL301psb6doAnausbCQFmSoIujme0jxcUG4YBBRHElMZOakSRy4S0o4qLvdtPTi4ihc3d0UHuMKNSkTkyaxU8TixbQGly+nRWny/ozV9OijbPsUCHDQBzjot7ZSFI4cCd1tvr8cyQ8+sMsLCylQra38eyid60MJx2D5mKG2YeqrLlzIGwsTdZqeTjFcsoSWcvB2Vq1iFZtFi+ycYnw8g5Wio61rWWu6TKdO5bn3eum6TkpijqYpdWe+x0CtugRBmJDInOJIYazEmBi6I10uCpRx41VV8bmxkcuMNWaeAa5vqrE8+CCb+jrn+gC6Xg8epNBOmsTBPzaWg/nMmQPPJ45GV/r+ipsPZV/OgCQjoidPMp3C5+N5iovj8ssvt/VVnZ9/6ineiHR29s6dnDaNRcOLivj5lBT799Sp3G5zMx81NbaU3OzZtp5sTk7fACkJxBGECY1YiiNFXh7nqUw+nnFhGldgTQ0HZK+XA3B8PAfqKVP4d3IyLaStW3tbJk6Ly1iNiYkc/JOTaUUlJfHZWEMDWXWj8b376/E4lM8aC3PTJqZ0rF9PQTOu4cREWsPHjvX9Xnl5vGk4dYrC6PVSTKOieL737aNwtrTQevZ6Gch0/DjPbXQ09zNpEvCpTzEo6sUXeSzmJqepKbRrVeYaBWFCIpbiudJfTltBAd2e+/ZxkG1uZm6hM3nedJo3SfXx8XxdWmpdeBkZTLswlonT4nJajc7I0XAxlOLmg3022MJ0nseZMylcJnLUWajcfD4mhjcLsbG2QHhUFF3Ll17Kggi7djHadKDzZq6rs1pQcTGvx4kTDHI6cICW/P33D60fpiAI4w6xFM+V/gIvTEHr667j/OGUKbSeFi2y7tLkZApmYiIHXtPH0OViXl5SEsulAaEtk3ON5hxtzjWYJtRngy1M53m85x4WQ7/nHhZCDy6a3t3NIuYAz11aGsUrM5PPtbU8Z/2dN6e1Z65rTg6XbdtGYe3o4A3O229zezt3ylyjIExgRBTPhYEGQ6doXX+97f8XE8MI0vR04LvfZf+/oiI+Hn2Un4uNpTCaXodGaJxiMRwBGi3ONZjGMJjAD7Zd5+dLSmgddnayg4aJ3l2xgoFKpk9jqPPmFMKcHEb8btrE4gctLezOcffd7Bnp9QKf+5wtxnC+LmNBECIacZ+eCwMloztFyzTDBZijOHWqbWt0++0c4P/jP1hZxViRxnXa3Ny7kbAJ+nAKhRPzfjg438CdgQT+jjsG3675PGDPW309cNdddG0aPB7g3ntptQefN1PZ55JLWFyhsZHnvLLSrnPllXyuruZ1qqujO3frVt70AOfmMhYEIeIRURwqg82fOUWrvJyDrKlM09JCd15xMa2MuDg26a2ooCVpIiY7OmjtXHppX2EYjcjRcDFcgTef37uXNxFJSXR17tzZWxTz8miFB0etArayT2ysbZ9VWUmrs7ycQv1f/0VXd00Ng38qK3ldOzutYAK95xoFQRjXiCgOlaFaNx4PO8ynpdGKqa2lOCrFAfgXv7CWjVK2ykptLS3MYGtnIjJcgTfRoQ8/zBxFY503NNhAmf4CeYDeNzgnT1LkfD5uIyODNydeL7fX0cHrZIqZHznC62is+bIy2+dyol83QbgAkDnFoRJqnqu7G3jmmdCVXOrrOfh6vXxdUUHXW3Exk+V9PpssbvLlALrmJHBjcAabYx0oVcT52cJC27vR7+c1MwLZ1kbL0OejCO7bRyFNTmZxhZ/+lIFR69fTopTrJgjjHhHFoWLy6JyP9etpNYSq5JKVxUhTkwfX3c1+g01NzGF0NtItLuYg7PXSYpTAjcEZKBhnqIE8J0/yuqSk0H3tdtMNa/I9AwHbJispiYE4d97J9U00rATcCMKEQtynTpw5iKbfYX899s6lkouzp6DPx0H3oosYANLQQKvymmtY07SrSwI3hsJALtiBIk7vuIM9JZ99ljcusbEU1LQ03qiUlFAItaYlrzXnhE1j5JMn+fcvfsHPn0+OpiAIEUtYLUWl1K1KqeNKqSKl1HdCvP93Sqk6pdTBnsdXR/WAnDmIgxWCHqqVYMRz7lwmkZsqN6WlnI+qq+O8VUMD14+EVIvxzmApHeba/uEPXD5jhq1uk5REMVSKy5SiC7W7m2KYl8f1CgvZniqSUmQEQRg2YbMUlVLRALYAuAVABYB9SqlsrfXRoFVf0VpvHPUDclp+AzXZddbbdFoJ27dzMH7ood6WghHPlhZaitOnW7fqQw9xoDadG5zRmOFMtRjvDGRFhmp8vGsXLfnUVApgXZ11nRo6O1m+z++3NzJlZZGVIiMIwrAJp/t0FYAirfVpAFBKvQzgLgDBojg2OC0/Y1FMm9Y33N7U20xJYSsjgFZCWRlbQi1fbgdFMwCnpbEiSksLB9XMTLpMd+wI3UVeGD2Cc01zciiKRiT/6Z+A738fmDyZwgfQ5T15MqNMU1N5Ddets6Iq108QJgzhdJ/OBFDueF3RsyyYe5RSh5RSrymlZofakFLqQaXUfqXU/rq6unM/EmdghmmyW19v549MRKiz3mZ5Od83hbrLy7ncGdBhBuDqaloZpgZqcTGtkNJScbeNJaECcLZuZZSpcYO/8IIt71ZbSyF0uWxLrYQEvt9fz0pBEMY1kR59+gcAc7XWywC8DeDXoVbSWj+ntV6ptV6ZkZFx7ntxhugbd1ggwJJsGRk26d6st3Yta3Nu2MCwfJeL+XJr1/YeKM3c1pEjFNdAgAPr2bN0wTU0jG1Hiwud4DQOgNe7s5N/p6Xx5uWuu9ifcdo0CudNN7HAQmcnb5Z8PnoQurvl+gnCBCOc7tNKAE7Lb1bPsv9Ga13vePkrAD8YlSNxBmYUFtLSa221bja/nyXZqqrYYgiw0Yb19RxsV63qvfyGG2yS+caNwLJlTPbevp1FwqdNE9fbWBNcSaesjNfWBDmZcm5VVfYzStn8RK157RYujIwuJYIgjDjhFMV9AC5WSs0DxXADgP/pXEEpNV1rXd3z8k4Ax0blSIIDM0y1lPR0Wy3lzTdp3VVVcVAEWGbsgw84v1hczHkpwM5D3nYb8PWvA+++C1x9NQfj2Fi6WpOSpGntWBN8nTdv7h3kVFjIv81zSwuLKrhcvDky7xlLU4JqBGHCETZR1Fr7lFIbAbwFIBrAVq31EaXUvwPYr7XOBvD/KKXuBOAD0ADg78bk4EK52crLOZ904ABdadXVtDQCAearmULeWtuyX1oDf/oT+yYWFdHqSE3lHFZnZ9jz2g4douFaVsaYoXXraNAO9D4w8GfGFc7SfM8+y+pEzmth8ksffFDETxAuEJTWOtzHMKKsXLlS79+/f3gbcVoQABXAqEBWFnDrrbQcjUCazhc1NRS+tDS65MrKqCymSkpUFN2mAK3QrKywueAOHQJ++EMeamoqNb2xkQbysmWh3z91il/vootCf2bckp1NUZw2zQqj8RbExtIjsHMnXaj9NZkeYZRSB7TWK0dtB+fIiPyuBCGMDPU3JRVtQuF0s1VVsY/eXXfRhdbVxYjFzk6KnFJ0nS5cSBFUiuJ54ACjUhcupGXY3c35qiVLGKUK2ITyMIji9u0UvLQ0vjbP27dT4EK9X1pKra+ooCguWsTl27fbz447C9JEpJpyfSb9Ji+P1+3DDzlvvGULb5ZM4v+iRWI9CsIERERxMLZsoTA65xJPn6aZZKy+AwdslRq/n7mKZWW2w4LbzajGT30qdBujMFBWBsyaZV/X1rL1o4kxOXjQilpNDbB/P5t+BAL86j4f40+uuYYG0+nTFMhZs2g9/vCH48SCzMtjZPD+/Zwb3roVuPxy4Hvfs30Yp04FXnsN+NKX+pb2G6wcoCAI4woRxYHweIC33uJg6ZxL9HioDkuX2kjEmTNpUZSXUy1M4ndFBZ+bmti3L0KCM+bMoSFbWUnRO3OG+h0TA7zxBoUtIYFf/YMPmEWiNcVQaxpWXi/z3pWid3jyZOr+4sXch7E6IxZjJZaX8+/UVFr93/42T4wJjGpt5cl45BF2w0hO5l3Exo0MphLLURAmDCKKA5GXx24IWVk2BD8/3zYHNkIZE8OBsbqafx85YpPBExKoHnPn0mL81rfC/a0AUM9/8xuKXl0drT+tOdXZ3s5D3r+f1p8p6xkIWI+x12sbfWjN92trqTEA+ySXlYX3Ow6KsRJPnOD1amriCfnzn2n1NTRQKLu6qPp5ecAXv8jPdnba/4Xly8MeNCUIwsggotgf/bUfuvFGWhFOobzjDtvJ3SyfOZN5ccuX914/Lw9YsybsLrfDh5klcuIEtSAqivWvm5v5OjqaQbPd3TwVHg9FMDraimEwgQAF9Y03OB3X1gb8/d9H8BxjQQGtdyN63d00haOj+WWSkug6NU2iU1LoX547lxYlwM+vWkUBlRQbQRj3RHpFm7HH4wF+8ANg2zZagkpxudvNUf6FF/oKZXV1bwFNS+PnZ87kc3p67/VzcwfuwDHCHDoEPPYYBeqxx/i6rAxYsIDjvtttu1r5fNQDI4a1tfQEu3pun/oTRCdtbUzNjI/vPcd46NBof9Nz5KGH6B6dO5fXxpT5S0qiy9SU86uvpznd3s48xb/+laLY1kaT2RSHd5b4EwRhXHJhi6IRQOdAlptL4fvlL6kIe/ZwsDt5kuJXW9u3XdCPf9xbQE1llKNHe1dIcQqrCdYY5UHUpFY0NvYWqNhYWoXNzfTwdnZS8AArekpxeVeXnW8cCmbO8eOPgffe4+edUaoRg3GP33MPcPfdtOozMtjFZMECBlIpxdczZgBf/Sq/UFwcv+CkSbyOx47RldrWJrVQBWGcc2G7T4PD6z0eRh+atkCf+xxw/DhVY8MG2+YpuF3QW2/RfZafT4vhwAGaSeXlfH77bZpf8fEUzIYG+i7HwOW2fTutv4ICCmBqKsf3+nq6TouLqcsDWX+BALXBBNcMhbg4fq6+Hnj9dX7WlApdsSJC3KnOsm9eL83buDiaxc4IYpOfumoVPQLFxbxDaG5m4M3Zs7zWSkVMIJUgCOfHhSuKzr56JkgiN5cDpNvNkMyyMg6AixdzneBapaa3YlQUTaHWVs45xsZyEAUoqu+/z/387/9ty8cBY9Kt/eBBpkuYBvKnTvFwAgFbXCeUIJqAGifJyfQgDqXeQ0MDv9KZM9SblhZ+/vRp6khEpGw481Gzs3mQpnqNSd5fvdqW+vvjH+lGra/nnYXbzWvZ1cX0jQULIiaQShCE8+PCFcXgvnqvvcY+elFRtiHwhx9ysCsv5wC4cWPvUmDBvRUbGtjN3eWy1seBA3z/zTcZ8uksH+fs1j4C1kWosmwVFYzzMWLW3W1FrbOT4mjiSpxi5+yvC9jKdEMtgNTVRQvVWJeBAKfu4uN5PMuXR1DSf6gbpOBSf8b1XVLCG6ClS/mFPvqIFuSCBTaQSixFQRi3XJiiGCqy9Mc/5midns4RPC6OVkFqKkWuvJwKs20b5xUfeIDRpqb/Ync3t9PQYC3K7GxrNZaW9hZMJyPgcnvtNRZc8Xo5LVZbC/zXf1FsfD5qvNdrRc1EkQJ8drm4npNgYTTrDxW/36ZrxsTw1MTHWzeusWLDnvQffIOUl9e3owbA/4GyMuDii+l7njGD75ti4WNg+QuCMLpcmKIYquC3CbGvr6eQmTDMM2c4YpeUUDCfeoqDX0sLB8jgUm/G8luzpq/wOgVzBDl0iIKoFAWxoYFBLt3dNlrU5BkaTCK+IVgQRxphgbqZAAAgAElEQVSvl/cYLS30Rjc38x4jOprHlprK3EYTkDNmothf6k2o63T//cxlaW2l2r/9Nr/Yxx/Tyhxhy18QhLHnwow+dVoBJSUs+BwTwzv+GTOAK6+kNZCVxR6IHR20HJWyQrhtG5f7/bQw//QnRpuaeqZGeAFGsCo1ap3at2/n2Jyayt0Yw9Xns6kUwVZfsLt0LDAN7QsKqCe1tTyGlBQu/+ADumjHNOk/lJs0+Dp5PLzrqK/nSY6OZi1ct5v/H11dDLIqKbHXXxCEccmFaSk6AyxMQMWqVTagwmnRvfIKA2SuuorCqZTtt2eCaerrOeIvXMjBE+Cz1hRcUzR09uxRiU4sK6OF2NlJ92RjI3dnku1NXnqk4Pcz/igxkcc5fbo1yg4eZOW0MSOUm9QciLlOeXl0ldfW8sapuJgnfeFCmr+f+xyv85NPWotTEIRxyYUpik5ychgMs3YtXzsthTVrmKKRlMR+iADNmjNnuN6+fdb8Mdv69re5/qZNVnBXrKDLbRRcpwADVDo7WV2uvd0KYFSU9QJHCkrZ0nDt7Ty+1lbGqURH837E9G0cdTweXsfgPopOqqpYHLy1lb5fU+R9zx56E1pa6FavqrKdNARBGLdcmO5TJ9nZtACM+8vpAsvLY2pGYyPzC2tqODhmZrKpYFYWB9O5c/l3enpvt5szgKOtjdGro5Csv24drcElSzjOuxy3OlERdoWdLlu/n6entdX2YL7lljGcTzR5qgO5tLdsoXvc7+f1TU1l5GliIj0HN9zA5H0AePVV/o8IgjBuibAhc4zxeChY999PU+WZZ4AXX+TjoYdo+V18MYVv8mRaiUuW8NHURDdaXR0HyKQkJnHv2MFRPjiAo7OTLSVyc0f8ayxbRoP0kks4Ti9dylgh40KNZMzxdXTQBfyNb4zRjoPTMELdrFRVccLW+KTPnrV1UE3FovJyWovt7Ywe2rJljL6AIAijwYUtik5LLji4wjSZLS6mIMbHs89eTAxw5530+1VUsNSXy2XNs7IyftYZwNHVxe2kpADPPz8i1mJwPVOAz3/7t5zqMvmH0dHD3tWoY8Q7Pn6MrcT+rj1A0XzgARux1N7OG5u2Nl73zk4q+fvv8wYJoDju3Cn1TwVhHDOoKCqlUpRS80MsD3eRruHRXyi+GdAKCmgpNDdz8Gtuppts927OF1ZW0looLeUgWVHBz9bX0/XqDODIz+fnXS7ORw4zArW/eqaHDtGVeuQI3abx8QyaHQ9ERTGzYUyKhg927QG+v28fJzkTErgsOppWYnIycOutwBVX8CRnZTFayNwYSf1TQRi3DBhoo5S6H8BTAM4opWIA/J3Wel/P2y8CuGJ0D28UGSgU/4476D6tqLBlvlpa2BMpLY2q4/NxuVKsfTq/z30D6a9c2DASvLdv52GkpfG1ed6+ndbivHmcJm1o6FuqLVLx+XiKn32Wj1FlsGvv8bBo+6RJttLApZfyZKan83/jjjuA736X7tOuLp7wmBhGGv/lL6EjjD2esLcMEwRhYAaLPn0EwJVa62ql1CoAv1VK/YvWegeAcTLc9sNgofjBA2d1NV1mFRX09fl8tBI6OpiysW1b6P0MNgCfB2VlHH/feceO2QsX2vy+FStoxL73Hr+is5JNJNPQwMbHWlM3Rs2VOpRrX1NDV6lZPm+eLQu0fTtTMIqLmcrj99OL4PNxWXV16P0GF6CfSBjBf+AB4Fe/4g3EV77COdY337Tui+hoW95o6lRbVtHM1WpNyzwpicsvvZRembNneZPS1kZ/u8/Hz0ZF8fybgDdzHS67jL+1U6c43ZGXxx/Ll7/MRtHr1wPXX8+G0qbixW9+A/zTP/H/4+KL6QVSij+umBje/Jw4we13dLBWodfL75aZyc/88z+zecDChTz+q69mnMHkyTyO48d5bAcO8IZq+nQGACxZwvNkOrN0dXHO+oEHOM5s305PxkUXMY5Ba37vtjZbUcnlAm6+mQPDLbewXNSvf80OP5/5DHNr33+f3iqleB6mTrXJzS+/DNx0EytvpaTwfC1axHPf0sJ9JiVxHLz5Zt7cz5vH//nUVH6PjAzu9+abOZXg9dpiKFFRPMbOTnu80dH8rlFRtn1bURHXN581U1Z79tAV9q1vAb/9LQcJrVlQRSnerI7AzabSA4yWSqlPtNaXOV5PB/BHAL8GrcaIsxRXrlyp9+/fP/wNbd7MHxTAi2NcaSYJEOA/q9a8oIcOhbYWndtxMn9+73zJc+Ab3+D/fXQ0fyemqPcNN3CTP/sZ0yujomx053ggKsrWUEhN5e/6G98Y45JvxrI3bURqavgj/sxnOHABFLbSUi6fP58i6HLRWlSKj08+6Z2zaLZrejUOkp6jlDqgtV45yt92yAz6u8rOZru15cvpOQF4zl55xaYzhQtzMwNQWBcv5pRGTIwdoKOiKGTHj4fOYQpVIDiY2Fhe04YGuywqym4vMdH2VWtt7S0UMTH2xkApG4EWF0dxb24e2nc1AhQXx21+85usAVlYaG86DG4351+U4lRRe/vQ9jEQwfsYKaKjud2pU3mzVVDA4v1a2zSof/3XAW82h/qbGsxS9Cil5mutTwFAj8W4BsDrAJYM8esMdJC3AngaQDSAX2mtnwx63w3gNwCuBFAPYL3WumS4+x2U4Py17Gxe7Jyc3j+Yri7+8/l8wCOPcAAI5jyFrz8OHaI+G4PVZAZ0dNB4eeQRCkogwJu7SMpRHIxAgKfU7+cN80cfhaEWqjPAyuvl/0JUFBsLx8ZyndJSLp82jQOasRYLC3lBjh0D/vM/gf/4j97bDa6vOlGsRTNHO3cu01JM482XXuKNQrhx9jtramKhf6fFAvCfz6TWhGIoYdzd3b0F0WzX0NbGwb2jwwqH6ehtkouDBaWzs/dxDuUYAArcpEmMpK+sDL1t4/Y3648Eo+WSMue/vp7/V3/zN7aUV1cX39uxY0TqDg8WaPMNBLlJtdYeALcC+Pvh7FgpFQ1gC4DbACwG8Hml1OKg1b4CoFFrvQDAfwL4D4wFwflrBQV0dQQXCNWa/9SxsVSqUea11+hNOXnS3rR6PHy+6CLeTO7dS+9HW9vozCeOxRyl30/j+tQpnvIxbU5cUEDLr7GRbqakJN74TJlCt9j69QysmTGDYllSwv+NAwd4sJWVHJh+9StamaZE3OuvDxzYM54xgm/cbB0dHGRra60LOpIw7lcAh7AUX8czWIEDmIdTyEANUtCATFTiGryHmShBFLxQ8I/Mw98NpX0jt72BHk31UBWlPfvzIhYdWIxDeA2fs+eitXV8/R/6/fxd1dVxvujQId5oREXZyP9hMpgotgGYFmL5KgAfDnPfqwAUaa1Pa627AbwM4K6gde4CXbUA8BqAm5Qa5WG5qgr493/nHYnJOdy0CZg5kwOkiXCZNo13JY88woHv9OlRPSxn0e9Jk/i7joqi699MU5ogWXNjORo5iqN1I+hMHTH/493drL89pqVEH3qIVsTs2bzOM2fyecoUutx27OD/hslNTU7m+pdfznUmTaJgdnbSWjTtxUpLB66vOl4xVmJaGi2tQIDLPJ7IFERDIIBDWIpH8DjexQ1oRRLKMRP1mIxuuNGCRHyIa1GF2dCIBm2D8fyIhhexOIGF+Ads6S2M44lAgL+tgwcpjpWVvGkNzhMfBoO5T58C8C8hlrf0vDcc/89MAOWO1xUAVve3jtbap5RqBjAZwFnnSkqpBwE8CABz5swZxiGBgQElJXaO6Gtfow/P7aYQGtO8tZUWwxg1lTWegowMHtrZsxQo0+PQWIxeL4VzpPMTExOH3mD4fHAKuPG+xcZSHE0a4JiQl8edxsUB993Xux7uZZexarlSvcsGlZTwzrWqiife9Or6/e+5PCaG1uTJk/bLAaNSB3fMMVZiSwvvyKKizt3lFya2Yx3qMBUpaMFxLEQUAAU/NKLQhQRYQZk4BBCFNiTiGXwT92JHuA/n/NCaVmFSEv9ubWUgE2CtxWH8rgYTxWla60/6HpP+RCk197z3OsJorZ8D8BzAgIDz3lBVFQcypfijbm5m0MCMGTa00wyGStG/F9x4eJRwFv1OSrItGjs7KVgej13XtIVyzvGfL2b8z8jg/kYLMz9vBD0+nvoRCND4GhOM1eN2M/Lvyiu53Fh2f/gDxdFYQobERB70jBm2SlJKCuclT59mXd3SUkbOjXcRDMZE8hYW8vdi5nic83gRShmy0IU4pKIZXaAVHwWNABT0BK1roqEQgAuVmBnuQzl/tKZbLBDg4FRZaetPKzXsm83BRHGg4Sj+vPdKKgHMdrye1bMs1DoVSikXgFQw4GZ02LKFg5oJifb5qDrbt1tTyWmaNzdTSMdAGJ1FvwGKlN/PQ0hJsXVEjWEDjEyQjQmWG+1yccYCjY21raTmzGFA4MUXj+6+/xtj9TQ38+Tl59ONali4EPjd7/p+LjsbePpp2zAyvuen0dFBMTQNqCdiA+IRDiQbS+Y8Bhx5A+gE4D7On3egJ5g84AP0OApSGxoKSgFR8QmYedUlwDvjIE8rDAx2O7RfKfW/ghcqpb4K4MAw970PwMVKqXlKqVgAGwBkB62TDeBLPX/fC2CXHiiHZDh4PKxL2tVlQyADAd7x+v3slFBUxMfHHwNf+hK7aCxezMF0FGqaOnEW/Y6L4/gbF0ctvu46jt0m7ScxcWQLgaelUSdGczbXbLuri/nx993H7xUdPUZdM5xVbq6/ngcwZ07vmrihBMB8rqPDhgTX1tJdCnB5cbG1NnNygB/8YHwFN0xQ1q3jzWVLC6eDTblBpXr3H59IREVxfNi4MdxHErkMNnQ+BODLSql3lFI/6nm8C0aF/r/D2bHW2gdgI4C3ABwD8KrW+ohS6t+VUnf2rPY8gMlKqSIA/wTgO8PZ54Dk5XE0njmTd/qxsVSduDiaMf/2b7YDggmeOH2aA2FCAqubFBXZAc/jGdHBz1n0e/581jD/7W+Be+/l2N3RQZeqqVVtXJHDxeT1traObh1V57HW1TGKtrt7DNMxQhVZKCvjxL3pbhLqmprPmZNvArGio/n/lJRE16LpvpKdPXhnDmFMWLYM+P73eQ+UlMQby8mTrbfi6qvpER8vVaEGIyaG48fPfsZxQwjNgO5TrXUtgGuVUjcAWNqzOEdrvWskdq61zgWQG7TsXx1/dwK4byT21YtQ5bYKCjhfVFdHs8h06PX7eSvZ1saE65QUJvh2dTHaJTmZv5rKSla28XpZBULrEa9esmxZaIFYt45jtym+YYpuAHZe8XztaxPM43ZboR0NW93p6u3o4MBUXt7/+iNOcJUbr5cHEBVlu5vExfW+ph4PrchJkziPkZHB/4kbb2RQTrBlaRL4J6ordRyybBnw85+H+yiESGKw2qdxAL4OYAGATwA832PhjW9CldvatImPzZs5h2gCKQIBDnQxMVSe1FTrUjX1Lj0eDphvvcVEwu3bqR6mLdEoD37LltlDr6mhXhuRSUgYfl6uCdwJZrQE0uzr1Cne1Y7JoBUsYKZgw7FjfH7kEeCaa3hNt2+3pcDcbi5LTOTdw/vv884klKt1IifwC8IEYTD36a8BrAQF8TYAPxz1IxptBuujt2kT5ww//phlhJ54ArjnHpYWUooCWVfHxG5jTXR22nqoR4/S7VZW1n9bolHg3ntZyrC9vbebc6QKVfh8fDhFcDQEMSqK242P5+n7cLjZsOeD+R8x1xWgm/zjj2nC7t7NSgovvMAqLtu28abnxAmWofr5z4Fvf7v3/9ZQOnMIghB2BhPFxVrrL2itfwEGuvyPMTim0WWwPnrO9fLygK1bOUf0xhs0wUxOjM9Hq9HnsyH5KSkcGOvqKJ7OqMMxGPz+8AdObc2bN37mQUy9U5OK4Xb3DhIKy/dwlnqLi2N1G4CRpJ98whuiykqastXVNl3DpCN4PCwuHdyfs7/C8IIgRAyDieJ/JxtNCLfpUO/Wnflqp0+zev3p07ZmYXU1TTDTsDA6mtEuS5ZwmdmeM+pwDAa/ykrqclNT7xzxSMbkQU6ezOdAgPcrHR28DFdfPcI7DA6WCRU84+ylWVvL90wE0/vv8+S2tNiuCd3dtCQbGvg5rXkRXn21d39OM2dpHlqPcbkeQRAGY7A8xeVKqZaevxWA+J7XCoDWWqeM6tGNNENt4+TMV2tp4WAXE8NHZiYn7oxJM3eurWTi8dhq9lFRjDo0+xqD6iUzZ9qynaYaTCQXFjGdhEwwjznmlJ7/qgUL2CVjRAmeT+5vfhng9cvP599uN63HlhY7P9zRwQjTmBgua26mVyE2lv8/hw7Z/61xnM8nCBcSg0WfjmIQfhgYrI8e0NuazMzkus3NtAhTU2kxmAbDXi8/V10dEdVKNm4E/ldPVqlpRxbJKNW7+1ZUFF2/s2axJ+S6dSOcjhE8n7xyZd/Xpk/bpk0Mtqmp4YFNmcL/E7fbeghMceLWVltlw+ulqpsygSNUuV8QhLFhMEtxYjGUu3WnNXn4MC2DlBTOG15xBQfGKVO4rKWF7aKuvjoi6ljeey+L8hQWcqw2rdkiscGwy8VjmzGDbl9TLtSUDh1xQQT6Rn9u2dL3dUGBtRr37rVpGQcPUgRN4M2MGbz+nZ10mxcVcT2vlyZubCxdqCNQi1EQhLFjYhb4Gw4FBRwod+6kW82EbwYCHPiUojjefTefvV52tI4Q99j119MwMToeiRiXaXQ0NcOUyjQNxE0qxogSPJ+clsao0fT03q/nzrXzzKtWsazO5ZfTMpwxgxneS5fSlJ07l0Xi3W6u7/VS0Y8d4xxjaysrH8i8oSCMGy4sS3EwnEWejYkVCNBdlpZmG3I2NHAkLy6m+jz/PHD77RHhIlu3jqmScXG2dnlbW+RYiybSVGuerqYm25IyI8O6U0c8FSN4Prm6miJWVcW8QvP67Fmq9de+xnXT0oB336WL1etlab/WVhZy+M//ZDTqW2/Zaz9tGpND33knIv4fBEE4N0QUneTlcQ6ovJyDYWsrR3Gfjy4ykx8wZw4DMBobOY8UHR0xLrJlyzgv53JRs+PjbUF5w1gJZGIip+k6OmyBoFOn+N6kSdQYk+2QlNRbQ0Y8FSN4Prmw0D5HRbFJcHw8b3zS0ymES5YA06fz5MXF0VVaVcW/8/LoHXj4YZ7sxYt5U7VokX0/Av4fBEE4N0QUDR6PbSDr89GqcLk4KAJ0kf6f/2PXffhhDp6HDrGySQSV7VqxgvmKq1bRk/fBBzbS03TR8Pk41rvdtifj+dBfeyqlaP0Zw+vaa2kJvvQSq6AdP247DZletCYK1eOhQTaiDOTeNtVramt5V/HOO1x+8iS9Au3tPFjTPcPMLa9ZA7z+uk3JSEqiuF51VUTMMQuCcO7InKIhL4+mVWurjRw03Xujo5nEb3LOnMndkyfzua3N5iKOcDHwc2XdOpuW8T/+BzU7Pp7xH/Pnc1yPjqZIXXYZoz1Nt6PzITmZ4qgUrcOLLrJTd62tFMRp06gpM2fSkFqzhvcZd97Je4vYWJvNMiqpGANRUED3aWEh8Pbb9kCnTrXdUZ56il/ke9+zHTNycvjFly7ll7jjDs5BbtgQMXPMgiCcGyKKgLUSy8qoFoEA7/4DAZpUfj8tgO3bub4zubujg8/V1TagwuS+5eWFRSBNR420NBown/40lyUk8KukptJCnDTJtsoJBM6t3ZRZ18wHZmZyPw8/zDnNu+6i2CYkUHwbG/nYuNH+bSzVJUsojldcAdx2G3VnxFMxBroGDz1EJb77bhs4dffdwK230gNQXR26NGB2Nr+E+R/Iz5eEfEEY54j7FKB4lZVxcIuKYqi91vQzKkWB9HppLX7xixxEKyqA1as5qre0AHv2MEkwOBeuvX3Eu2UMheCOGocOUdPLyphBYqrf7NnD9I3u7sG3GRvLe4aurt4dM6Kj+TWd0a7TptEKrazkqZozh+Vjly2zNbXLyrj8+98f5fZQoRL0g9/v7uZ1dAbfmOIOP/4xb4rWrrWFvNesoYl8333WJ93QwACcCHChC4JwfogoArT86uo4AKam8tlUsJk1ixNdMTG0EFpbQ0cyVlUxz+2qq2zuW20t8Mtf8nWYk7iDRfK119hVw0yhmi5ZoTCJ9abmOUAPs9Y2mtQIbWMjT2FzM7f5k5/0Fbz+WmCNCsE3KcHXwJmq8eGH9CPv20dX6uLFXGfHDj4XF9uUjfb2vtWR2tpoCj/zDE9OcHsyQRAiHhFFgPM/y5dTwB58sLc1kZ3NQTEriwWh8/KY1P3JJ9a8OnCAZld2NguHX389l3d2smtGQgLVI4IiEg8fprFjYkeAvh01nIn/pi6pEUWt+bXmzOG06lVX0RpNS7MWoLEMw8pg7ZqcNzjmuu3eTatxwwYKXX4+LchjxxjaawqAa83nSy/l9a2q4p1BXt6o9NMUBGH0EVEE+rcm+isgfuONtC43bODgFxtL0dy+nSH/F19Mi6KoiMKYnEyTLIJKfpWV0Qh2uxn40tHBcd1ZGs7Zgio9nRZgbS3XBWz510svpbiuWAE89tiYfo2B6e/6Oa9Bf82Fp0zhug0N9A60t9NFmp/PBP6FC+2N1IYNvMN4+GEm+o9xP01BEEYOEUWgf2vCLAc4+bZsGa3EU6c4IJrBLzOTA+aZM/Qx7ttHq6Gmhp/t6qJZZSzNCLAc5syhq3P6dIphQwMT6U2Khc9H63DePM4PfvrTfK+mBti/n01DtKbb1O3mtr7ylXB/qyCGUgDeRIl6PHR3ZmXZm5yiIp6YzEzb/aS5mak5iYkUQefcsfkfOniQ/xdz5kgzYUEYZ0j06UDtpIwVkZ9P19nbb1PYPvqIA2xZGV+73bQ0pk/nYGrKf6WkWMHs6qIFYrouhBmTtjFjBsVu6lQGYN5zDw2fgwdpER49Sq+iSZfIzAQ++1lGiS5ZQsMqLY36EHZXaTDn0q7J9M989lngvfd4/Ts7ed1NmGxbG7exZUvvG6m2NjYcNte6vj4s/TQFQRg+YikOZE1s2mQT9ZcsseW8mpqY2N3dTX/i0aO2g4IJz2xuZva8SedYvpx+ytWrw/t9ezBpG9u308hpamKKxsUX9y3GvW4d8MMf8u/BgmgiiqHmCjpTcqqqeP327qWAtrbyy545w+UA6+JWVtobqc5OBuFccQWfzWRscTHdrKHakwmCEJGIKA7WTsoZrt/cTAWJiwM+/hi4/35Gns6cSXdaZiZ9i1ddxYCb1laWbsnIiMhKJ0ONAnUKaEQF0YwUJiWnqooCGR/P/4vLLqMJHR9P689EJmtND8GcOfx8QwNvfvLz+bepoTvG/TQFQRg+YRFFpVQ6gFcAzAVQAuB+rXVjiPX8AD7peVmmtb5zxA9mIGvCWBBFRfzb7+egGBdHy2HPHk66ffABgy/y89mJPRCgZVhczHVjYznArl8/bgfGMU2jGC3MvKEzTcJZ3i8+njc/8fF83+WiZZ+fT19ySwutQK+XVuPJk5yQnTOHj/nzpZKNIIxzwmUpfgfAX7TWTyqlvtPz+tsh1uvQWq8Y20NzYCyI06c5+MXG0lpIT6f7TCnmpJlIVRN92NDA6MUPPuAAW1JC8ZRIxPASKonfXGOfj8IIcI7QdEX58pcZXeR28yZn5kzOFS9aNK5vcgRBCE24Am3uAvDrnr9/DeDuMB3HwJgms93dtAxbWjh4lpfTcigstPVOnYEXZWXA735Hd2trK8X0k09610cVxpbgtBsT+GKusWkdpRTnCE0KxtNP87qePcvXZWUUz/Jy3hBJAI0gTCjCJYrTtNbVPX/XAJjWz3pxSqn9SqkPlVL9CqdS6sGe9fbX1dWN3FGaJrP/+I8MmFi9Grj5ZrrVUlMpjO+/3zuC1eNhZnxTE//u7ORAW1fHgVTqYoYH502LCXwB7DV2ufiIiWE4bmsrb2JOnmQCpikaqzUji+PiuJ7zJifMheAFQRg+o+Y+VUr9GUBmiLcedb7QWmulVH+Ni7K01pVKqYsA7FJKfaK1PhW8ktb6OQDPAcDKlStHpltgcKrGrbfSLZqSwsCZWbNY1DM2tncE6/vv06KMjqYYTp/OYI2GBg6k3/rWiByecA4EX8u0NODxxxn4VFBgy/tNnWrr3cXF0eX917+yak1np62afuIE3eNLlvR2iQ9WY1UQhIhn1CxFrfXNWuulIR47AdQqpaYDQM/zmX62UdnzfBrAOwAuH63j7UOoVI2mJjYEnDKFy6ZMYRHRPXtoQRw9ylpnxgXn9XJeqr7eZr5v3BjakhAr4/wZ7NwNVKt20ybODV53HZM0776biZg+H13mc+Zwu2lpFMqUFLpRW1r4MFZnf+5ZQRDGFeEKtMkG8CUAT/Y87wxeQSmVBqBda92llJoC4DoAPxizIwyVqnHsmE3JAPh89iytxRdfBB59lIn9s2YxFSMujgLZ0UHXa3U1g29C5ayJlXH+DHbuzLU8epSWvBG3nBzbZbm7m/mHAN8rL6dwRkfTrWqq3ADcTmIiy/lo3beijVSxEYRxS7jmFJ8EcItS6iSAm3teQym1Uin1q551FgHYr5QqAJAH4Emt9dExO8JNmyh0zkdGBiNPKyrsA6CLzeNhcn98PC3KKVM4PzV7NgfWW26hZTFlClMAnJbEYFaGWJH9M5Rz53YzKObiizlP2N7OnolpaRTC1atpLbpcvIlxuThv6PPZ+cUzZxg4VVnJ7XZ0MBrVVLoxFW0AqWIjCOOYsFiKWut6ADeFWL4fwFd7/v4rgMvG+NAG5i9/sX87c960Br75TVbGXrCA7rX9+ymgzc10xz39NCMXY2LoUs3NZfI/MLRODmJFhmao527qVOD3v+e8YH09rcf6er5+5RWua9yiVVW0JrVmmR+fj2I5dSoF01RKN8n51dXc90A1VgVBGBdIRZvzxSlUWtMtOmsWB83ycg6YPh+thiVLWEQc4LK2NuC554Dbb+dnB+rkMFg/wAuZwbpgOM/d00/zBsU0j9yzh/OFUcWQPgUAABYRSURBVFFMlzE9sjwee43i4ih4mZmcH772WjahDGbzZl7f/qoiCYIwbhBRPB+cg63plPHZz9Jd9thjfMTFAe++ywjGvXsZ0RgI2FZTR4/SWoyLG7iTw2CW0IWMM4DGWOfTpvU9d1qzAINpHKkULcL6elrzZ85QCDMybPRpTAwtf4+HNz5xcUzfCIVUsRGECYN0yThXPB5GkLa12UT90lKb//bjH7N11NGjtBoLCmzpN7+fD6+X23n5Zc51dXeH7uQwUAcPoXcwlOlkUl3d99zt308h9PvZETk5mdfG72f+aUICt9PQwO2aeceiIp7rwsL+u2sIgjChEEvxXMnJoQWyapVtEwRQ2NLSmLLh8zGiMTbWlgiLi7PL4uP5uriYg/aGDb2tPzNfmZs7eD/ACxlnL0RTYq+1lbmgu3bZc1dWRpELBNgvq6uLbtPmZkYLNzfb7spuN63EzEyWeBMrUBAuKMRSDEV/0Z4eD6MMU1IoaCdPcrlSfF1WRrdcRwdFb/58uuN8PophdDQH6s5OrlNWxkHYaf15PAzayctj0vhQ+wFGOiMdQevcXqhqNU4rcv583ohkZNC9mpnJrsnr11P4TGeL0lIWaVi5ElizRgotCMIFiFiKoegv2jMvj/NPLpe1MqJ67isKC3tbHNHRdL95vaxq86lPMarxxAm67AAK6Sef2Good9xB6zAvj1ZPYiLwox9NjMAa08T36FFbRH2429u9m7mDu3b1dTH/6EcUxaee4jVasYLipxQt+tbWvufWFAevqaG1KBa5IFxwiKUYTH95b2b5rbey6sl99zH38OBBit977/H1ypUUvcxMuuxcLgrjjBnMk1u/nlGPWVlsShsdTcvxj3/kfNjWrZzj2reP+Y79FRAfzPKKpNxGc+5MSbzc3JHZ3iWX8Hy1tVkXM8DApq9/nfvJzqbQhZr/Da5balpITZ7M5+B8UkEQJjwiisH0Vzg6VNm3UO/fdBNFc/FiCuKsWbRGurrYaNjMcZ0+TUtx8mQ+t7UxSKekhNaMx8PUjmBhdroMd+/uXzQHe38sycvj96uo4LnYunVoYtOfsDuv0ZkzvJkw7mXT9WLXLqa9dHWx6pDHQ6Grr7fzv48/TqvQbLOsjOfe7eZzaWlknD9BEMYMEUUnA0V7OueoQs3vOd8/eZJVbpTiAOxysZVUfT3nCYuK6G5tbOTcYnMzB/IdO7huYyPF48SJ3taiEbqcnP6t2R/8gCIRKXU4zTk1BbXT0ngTMBRrMZSwB1+jG27g93vySeCnP6UVPmMG3Z9HHQWQPviAAhoIcP/O+qeAFVOfj+fctAjLzx+5cyEIQsQjc4pOBrIGB4tCdEZCfv3rnA+86CKKXHU1A3BiYzkwJydznuuKK4ClS2nNfPQRP9PSwtexsXw+dozCu2ZNb5ehKTm3bx/wta8Bv/iFFRFTqDo2ltaps3rOWGOsxOJifj9TR3TrVhYv6G9uMTgX9OBBBr4MVNz7qqto7QEUSr+f20lK4tyt388blyNHKNIpKcAbbwDf/jajiSsrbX1TgJbi6tWje34EQYgoRBSdhCoCDpxbZZK8POCdd2z7ofp6ioLWNn2jsdEKpOHwYSb6d3Swu7tJIne7e6cYxMZSYJKTeZzV1RSCv/97Dv5z5wLbtlFwSkpo9Tz//MACZHCWrhup4J6CAh5jczO/D8DvVltL0SstDb0/I6anT/N7Hz7MYBnnNWpvt6X3nnoKWLuW57izk98lNZVl2m68kRb64sWMPr3xRuDNNymAxkU6EtdeEIRxj4iik+HmpBnrJiuL81jp6Yx4TEzk4NzcTJHzeBhMExPDQbi8nJZjdDRbGAVbKzk5NsKypIRiW1RkCwJ0dACvv06rZtYsBvaUllI8MzKsu3IwazE46nYkRNKc01OneFyFhawUExPDIJiWlr5Rvk6Xa3Exv2NyMt3LP/mJPZZHH+V5MSkuu3fzhqK2lu5Pj4dzhm+/zXPV3MzPvvACo3uB3tGqEyHKVxCEYSFziiOJce2tXUt3XFwcXXoJCbRa3G4+kpIolFlZwBNPsIj4/ffb9kPB85Y7d1qXYW2t7c1YXk6h6eri84kTdAHGx7O2Z3ExhaC+ntbjQISKuh2pYB3TcWT9es73bdjA+b+kpNDznk6Xa3Q0hRPoHfhSVcUC31pTEJXijciZMzyPMTG2gpDLxWjh66+3QtvdzXNk5n0loEYQBIgojhzBASBm8K2v54BvBKy7m6LW3s5BfssWG0m5dCkFI7hllSk6XlLCQJIlSyi0Xq/t2KAU3bKTJzM1pK2NwjBlCoNbmpsHDrgJjrrtL5hnuOfHbM9U6wmVHmFcro2NFLpAgJ/3eu2xbNnCZZ2d/IzLxZuQpCTOFa5axXPhcvH7mznIhgYK5dtvc742P3/8FkQQBGHEEVEcKYKLUxcXc5CeNYtCEBVFF2pSkn3U1bEH42C1TY2l9dOfcl7szjv5eb+fwqE1Hz4f8yXfeIOi6/MxAjM9vW/EpzPdIVTUrcn/CyVa53t+2tqAQ4c4z7l1a//f+6GHaD1fcQX7US5Zwu/wqU9Zwc7N5Xdsb+cNgc/Hx4kTfDZtoRobbfUhc1OxejXnZtev536eeUbKuQmCAEBEceQILk7d3EwhrK1lYMedd7J9UVGRfTz+OK1DZ5eHtrbBcw+3bWPOX3c3B/5AgPuKirJCkJRkS8qdPEkra9s2K4S5uZxbMxV0nBGdAL+HscIyMxkBunnz+eUXOucIS0tpNRcXU8yA3lG+zoLrZ87w/DU385Gfz/WeeILuaMDmFcbF2f0HAjb/MD6ewu60wNevBy67bOQEXxCECYME2owUTkvD9NczmIjG4EhGp5CWl9tuGqEiHp3ux8OHgWXL+HmPh9swaR6BAAVl6VJuz3SGX7bM5jxmZdFS6+pigrvPR9Hp7qbLtayMVqjpGmEKDpgI0MGiMYMDdpxzhJMn22Ch/HxagoaDB/ldPvzQ9qaMjmZz39mzWcN0+XLgn/+Z4mpEt73dRusCnGM0xdfdbn5v02dxsP6VgiBc0IgojgZDdcUFd3lYscJ2eQjGWHOxsUxT8Ps5iHd0cKDv6qJwVFQwiMfjoUg2NXH+saCA22lv5z5qapiecPQoran0dNutwynqJ09yHY+H6w8mIKGaIhcUMDDGRH8mJDBK1OXqXQfVnIfPftbWKTWdL0wt04cfpjt1zx66U2+4wRby3rWL36OkhAIeGwt85jN8NkE6WkvnEUEQ+kXcp5FAf6XlDM45v5ISukKbm7n+pEkUNaVo5Zlgk7NnKZheL58rK+mOPHsW+Phj7ic6mu+dOcNnU+vTzGEaV6PLRevylltC1wx1ukpzc205O7PuQw+x7ut999m6sXFxFO2cnNDdLkLVKTUWpzkHhw7xc845ULebgh8dbV2uzupDg1UmEgThgkYsxXBhcgAfeGBwd55zzq+2lq7B7m5bvaWriyJg+jOmpgLXXstqN243rcrycooGwOf4eAajGMuptZXbc1pMpkh2eztF9fLL+x6f01W6Zg3FqamJxzJ9OudNGxpCz1kuWsT1o6J6d7sI7lNp5jSV4ndvbuZxt7eznN7atRR2n88KuMn1nD9fgmgEQRgyIorhIrgkG0CX4FVX9XXnOa2bOXP4AOwc244dtjrLunX8XHY2RaSykp/z+bgNg8dD69Dv53ptbRTcf/s3HkNmpi2SbVyue/cyaKitjcEwTzzR21VaX899ZWRwPtPrpdv0hRdYacfMrTrnLGtqGFFrStdNnw4cP871lOJ2zHynKZTu9fKhFC281lYGzlx6qQigIAjDIiyiqJS6D8BjABYBWKW13t/PercCeBpANIBfaa2fHLODHE2c825vvslWU3v32kCb2bN7B9v0N9CbObhQVubevSyC3dRE68pEkhqM+zQ+nnNucXHc3unT7NaxaROPsb2dwh0TQ3HOzKT4VVRwvQMHaKk1NFD8TIWdlhYKXnIyI21ffpmfNce8erUtQ2cKcRtLr7DQumNNC620NIpgdDQfWtMi7ujgw8yHCoIgDINwzSkeBrAOwHv9raCUigawBcBtABYD+LxSavHYHN4o45w7u+wypmuYqjbnkjc3UAHzpUvtPhISKCQuF8UtPp6vExMZmJKUZNNHvF66Kk1y/dy5tEwvu4yW3Ny5PMa1a4GXXqIYFhdTtM6cAe66i/OGl13Gz196KcXMJNt/85vWjXviBBPt/X7OOS5YQOvz85+nFf2lL3HZ5Mnczt/+LcXU6+V3iY/n92lpkd6HgiCMCGERRa31Ma318UFWWwWgSGt9WmvdDeBlAHeN/tGNMoMlyhvX5FAG+P6CRvLzabUZd6PHYztE+P0UFYCpDiZn8vvfp4V65ZV87/vf57EcOGDbKcXHUzDb2lhxxuOhuL73Hi1M4y7t6mL0pynYPXky8POfU+T27OFn9+61VXb8fgb/tLVRPHfv5vOOHbQap02j8C5bRmswKYmi73IxinXGDOl9KAjCiBDJc4ozAZQ7XlcACNnHRyn1IIAHAWCOmW+LVEIlypuuFwDdnHv3Dq2Ad3/W5MsvA6++SlHUmlagy0UxMcKYkcEUEIDi9sILdKG63Vbc1q61NVoBit22bTzG48dtDVafj7mFSUkUsaoqVusJBLhufT2tuTff5DanTeP+TY6iydEsL+fnr7mGx28KfTvbTSUnc7utrdxHWhofpgSc5BsKgjAMRk0UlVJ/BpAZ4q1HtdY7R3JfWuvnADwHACtXrtSDrB5eglsUlZdzkK+vt+XhUlKG3u4pGKfAzZhBwTp9mq/nzeO+T5zg3wsX8jM5Odzv1Klcv6uLwr1lC/P/go/1zBmK1KWXcluZmRSuvDyK6HXXUTBNMn11NZ+9XuY9JiUxF/Hxx3vnaB4+TKE8e5ZWZHs7Xb+NjUw9KSqi+/a+++wxhcpFlLlFQRDOk1ETRa31zcPcRCUAR7kTzOpZNr4Jtu6cifKmPFxSEoUnOD3CpHD89rf9t3PKy+NnTeI+QGHJyqIFVldnE+JNkYDsbGt9NTbSqgsEKGIxMcxXBIDvfpfbVMr2RzQ1V7WmiC5dShH0+XgMgI1wBRgNu2iRteqcRQlMvdiKCn6mu5uvq6tthOzp00zaD85FNFan9D8UBGEYRLL7dB+Ai5VS80Ax3ADgf4b3kEaB4Ko2q1fbWqih8gFbWmhthupB+OyzfH/58r77mT+fwSw//KGNFjWiu2gRRcnrZYNkpShK8fEsWH7qFD+/ahXnGGtqWKz74EHO6QEU2507Ob/o9dL1aVJAAgE+R0dTeE+c4PxgqD6Rhw/T2kxK4ueSkmypN1fPv2tJieQiCoIwKoQrJeNzAH4KIANAjlLqoNb6M0qpGWDqxe1aa59SaiOAt8CUjK1a6yPhON4xYaBI0jVrKCBz53JO75ZbQif4794NPPgg3ZLBeDzAvffaxPq5c+02jKC88gqLkgO2SXJjI/DII8CvfsVjiI2loGZl8e/aWro2u7r4ucJCWomhcLkodOXlnFPMzrb1SU2fyMZGuk1NZGxyMkVehE8QhDEgLKKotd4BYEeI5VUAbne8zgWQG7zehCR4rtFgimR3d9MK9HpplcXFWUsvVL3RYNeqmTc0ifXz5vUtEpCdTXE7e5avW1q4n127bNRpRQW3/frrTLQvKqJlmZzMY4+PZ+DL2bO2KAFg3acJCRS/a6+lIJ461bsowYoVIoCCIISNSHafXlgMlqCflga8+y6bBp84QesxeF4uObm3W9SJmTfs6KAL08zDmTk4j4diN2MGxTApiQK8YAGjSZ96imIVCPBYamooalddRetw6VJaoXFx3EdKio1+TUzk60WLGNxTWkpXrMz9CYIQYYgoRjpG8EzwS1wc5+uqqvi3c14OCF071eOhYN53n52vbGhg54ng+qomUrS1laJYVERha22lhZeYSDeo1wscOcL0ithYCrXWtgBATAzX8/n4OiqK4mncwxIQIwhCBCKiGOkYt2phoU3EB/h6+XIGt7jdA7dCGmi+Mri+6vXX997//Pl8fucdWnh+Px8m6b6lhdZlRwfwhS9QkBctolg6RVcQBGEcIKIY6Qw2t2ZSOkLNRRq36JYtdLv2t85Q9gPY1JGyMuuGNcXCW1uBt9+2QpmWJjmDgiCMO0QUxzuDiVleHt2bwy2Y7dyPM7fS4PWyys3f/E3olBJBEIRxgIjiRGYoUannQyghzs5mrVLpaC8IwjgmXF0yhLHAGZVqBGq0kI72giBMAMRSnKiE6sYxmu5MySsUBGECIJbiRGWgiFNBEAQhJCKKExVxZwqCIJwz4j6dqIg7UxAE4ZwRS1EQBEEQehBRFARBEIQeRBQFQRAEoQcRRUEQBEHoQWnTHX2CoJSqA1A6CpueAuDsKGx3vCHnwTKa5yJLa50xSts+Z0bpdyX/S0TOgyXsv6kJJ4qjhVJqv9Z6ZbiPI9zIebDIuRgecv6InAdLJJwLcZ8KgiAIQg8iioIgCILQg4ji0Hku3AcQIch5sMi5GB5y/oicB0vYz4XMKQqCIAhCD2IpCoIgCEIPIoqCIAiC0IOI4hBRSt2nlDqilAoopS7I8Gml1K1KqeNKqSKl1HfCfTzhQim1VSl1Ril1ONzHMp6R35T8pgyR9JsSURw6hwGsA/BeuA8kHCilogFsAXAbgMUAPq+UWhzeowobLwK4NdwHMQGQ35T8pgwvIkJ+UyKKQ0RrfUxrfTzcxxFGVgEo0lqf1lp3A3gZwF1hPqawoLV+D0BDuI9jvCO/KflNGSLpNyWiKAyVmQDKHa8repYJgnB+yG8qApEmww6UUn8GkBnirUe11jvH+ngEYbwjvylhvCGi6EBrfXO4jyGCqQQw2/F6Vs8yQegX+U0NiPymIhBxnwpDZR+Ai5VS85RSsQA2AMgO8zEJwnhGflMRiIjiEFFKfU4pVQHgGgA5Sqm3wn1MY4nW2gdgI4C3ABwD8KrW+kh4jyo8KKVeAvABgIVKqQql1FfCfUzjEflNyW/KEEm/KSnzJgiCIAg9iKUoCIIgCD2IKAqCIAhCDyKKgiAIgtCDiKIgCIIg9CCiKAiCIAg9iCgKfVBK+ZVSB5VSh5VSv1dKJfQsz1RKvayUOqWUOqCUylVKXdLz3ptKqSal1B/De/SCEHnIb2r8IKIohKJDa71Ca70UQDeAryulFIAdAN7RWs/XWl8J4F8ATOv5zP8P4IHwHK4gRDzymxoniCgKg7EbwAIANwDwaq1/bt7QWhdorXf3/P0XAJ7wHKIgjCvkNxXBiCgK/aKUcoG93j4BsBTAgfAekSCMb+Q3FfmIKAqhiFdKHQSwH0AZgOfDfDyCMN6R39Q4QbpkCKHo0FqvcC5QSh0BcG+YjkcQxjvymxoniKUoDJVdANxKqQfNAqXUMqXUp8N4TIIwnpHfVAQioigMCc3K8Z8DcHNP+PgRAE8AqAEApdRuAL8HcFNPlfvPhO9oBSHykd9UZCJdMgRBEAShB7EUBUEQBKEHEUVBEARB6EFEURAEQRB6EFEUBEEQhB5EFAVBEAShBxFFQRAEQehBRFEQBEEQevi/Ti4tJjX5/u0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff3a414e310>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(7,3))\n",
    "\n",
    "ax[0].scatter(X_spca[y==0, 0], X_spca[y==0, 1],\n",
    "             color='red', marker='^', alpha=.5)\n",
    "ax[0].scatter(X_spca[y==1, 0], X_spca[y==1, 1],\n",
    "             color='blue', marker='o', alpha=.5)\n",
    "ax[1].scatter(X_spca[y==0, 0], np.zeros((500, 1))+0.02,\n",
    "             color='red', marker='^', alpha=.5)\n",
    "ax[1].scatter(X_spca[y==1, 0], np.zeros((500, 1))-0.02,\n",
    "             color='blue', marker='o', alpha=.5)\n",
    "\n",
    "ax[0].set_xlabel('PC1')\n",
    "ax[0].set_ylabel('PC2')\n",
    "ax[1].set_ylim([-1, 1])\n",
    "ax[1].set_yticks([])\n",
    "ax[1].set_xlabel('PC1')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "标准PCA效果不好，处理后不能应用线性分类器。\n",
    "\n",
    "再来看看RBF核PCA的效果能不能让我们满意："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_kpca = rbf_kernel_pca(X, gamma=15, n_components=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcsAAADTCAYAAADu8DgFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzsvXt4W9Wd7/1dlmzJl/gSx7c4Mbk55IZxaAKBFggESpkB0ppA23NeYIb2ZRqa4Zl3hocyXNo8pWVoh56Tw0mTDmeGM5TpFGgmhVCgFAKBtiE0IQTj3MiNOHZiOxffbcm2vN4/vlpdW7JkyXFsyc7v8zx6JO299t5rK5G/+l2X0lpDEARBEITopCR6AoIgCIKQ7IhYCoIgCEIMRCwFQRAEIQYiloIgCIIQAxFLQRAEQYiBiKUgCIIgxEDEUhAEQRBiIGIpCIIgCDEQsRQEQRCEGLgTPYHRZNKkSXratGmJnoYgnDUffvjhKa11QaLnYZDvlDDWifc7dV6J5bRp07Bjx45ET0MQzhql1NFEz8GJfKeEsU683ylxwwqCIAhCDEQsBUEQBCEGIpaCIAiCEAMRS2Hcs2EDsHQpUF7O5w0bgOpq4LbbgOJiID8fWLSI2wVBECJxXiX4COOb6mrgsceA3/8e6O0Fpk8Hrr4a+PWvgexsoKQEaGkB/u7vAJcLaGgAlALcbuDTT4Fvfxt44QUe29IC5OYClZVAVRVQUZHouxMEIZGIWArjgupqit3OnXyvNbd9+CGQmQl0dwNNTUB/P9DaCvh8gNdL0ezpAfr6gM5O4He/Azwevk9N5b7Dh4H77xfBFITzGXHDCuOCjRuBffuAQIACqBSFEQC6uiiQLS3c3tvL7VpTDAE7vq0NaG/n9o4O4IMPeM6NGxNzX4IgJAdiWQpJT3U1xaq2FigrC3WLmn2/+AXQ3AykpQEpKRS7lBQKndZ87u/nGCOiPT10waak0JI0+P20KpWiyP7mN3z/7/9OizQnB7j2WmDlSrE2BeF8QSxLIWmprga+9S2K4+uvU7Cam4Enn+S+6mq+bm4GJk+muPn9Vhid9PZyu99P8QT4bITUbDPbzdhAgNbm6dMU664uvv7tb4GHH+YcBEEY/4hYCkmJEcKPPgImTuS2bduAEyeA/fuBO+8E7ruPFmFeHjB3LpCRQaHz+Tg+ELAJPEYYAVqShv7+gcJqjnViLNCuLsY/GxsZH12//tzfuyAIyYe4YYWko7qaQtjUBJw5A5SWAhMmUKg2b2YCTkeHTcL59FOWflx6KfDee9YFm5JCt6zbTesQ4GuAwulycYzHY+OX5lingCrFB2AtzpQUWpjPPw8sWwasWDG6n5EgCKNLQsVSKfUlAP8LgAvAv2qtnwjb7wHwcwCfA3AawFe11p8F91UA+BcA2QD6ASzWWvtGb/bCuWTDBmDtWmaempjhjBnAqVPAnj3MaPX5KGZaU8yU4rbaWgprZyeFz+tl7aTbzcehQxQ3l4vX0prHpqYCU6dSkDs6aJl6vRTh7m47Li0t1AJ1WqxpaSxXmT1b4peCMJ5JmFgqpVwAfgrgegB1ALYrpTZprfc4hn0DQLPWepZS6msAfgTgq0opN4D/AHCH1vpjpVQ+gN5RvgXhHLFhA3DvvRQ7v5/blLKJN1pze3c393V309J0uxmvbGmxYuZyUfA6O2ltlpWxntJYg4AVY4C1lyUlwPHjtD6zsvja7ea1TRmJMwHI4PEAU6ZwPhs3ilgKwngmkTHLSwEc1Fof1lr3AHgewPKwMcsBPBt8vQHAMqWUAvBFANVa648BQGt9WmsdFmUSxgrf/S4FD7Auz0AAqKujqOXmhlpzpiayq4v7+/oolFpzf38/t23fzhjnddfZmklzDQBITwfmzGG80+vl+74+vs7LAy67jNZmpJhmfz8bHbjdQEEBsGsXsHo1cPfdfJbEH0EYXyRSLEsBHHO8rwtuizhGa90HoBVAPoDZALRS6g2l1E6l1APRLqKUukcptUMptePkyZPn9AaEc8ORI7ZUo7/fJtdoTQELBIDCQrpMAY5JSaELtr099FyBAAXP7+dzSwtw++08Pj3dum+9XpZ/FBcDRUXARRcBF1zAeVx0ETNwb7iBXYDS0ymK6ek2FpqSwiSfo0eBTz5htu7/+T+0Tp0Zu4IgjA/GaoKPG8AXACwG0AVgs1LqQ6315vCBWuunATwNAIsWLdLh+4XRJVLNJGDdrc7aSIAxwZwc4PLLgR07mFTjclEonSUght5e6771eCiMmzbRJbt/P6/r99MqNLHI1lae86mnOLfmZlqWAOdixDIzk9ZpS4t1zRrXsNvN2OdvfgNceSUwb564ZgVhPJFIsawHMNXxfkpwW6QxdcE4ZQ6Y6FMH4D2t9SkAUEq9BuASAAPEUkgeTDlIXh5jfcYCKymhhebMOgX4uraW+/fupRgVFVE8jxyhMPp8tiOPwSTm5ObydV6etTRnzOC2ri6gpobu3MpK4Bvf4LENDcBbbzFBqLKSgmrO53bT8uzsHBjD7OuzJSrvvsv7yciQvrKCMF5IpBt2O4BypdR0pVQagK8B2BQ2ZhOAu4KvVwB4W2utAbwB4CKlVEZQRK8GsAdCUrNxI4UrL4/Wn3n9uc9ZK89YmM74pMl47exk1ulNN9EqXbiQgpiSQreqEVqXi5ajEcycHLbCM3HJ9HSK3vz5tBJra1kv+dBDdKPm5DAr98UX+XrePGvNdnbSmnSKusHvt25kk5kr7lhBGB8kzLLUWvcppVaBwucC8IzWerdS6vsAdmitNwH4NwDPKaUOAjgDCiq01s1Kqf8BCq4G8JrW+tWE3IgQlepqitC2bXSfnjxJUczLAxYvpljl5DDuePPNdGH6/VaITA1kSgrFb84cJv3MmsX3ra22KYGzM09GBsXt+utpiX76KcUSoGBmZfG5pobHX3UV8MYbLD9JTeX55s3j+RsbgUcf5WokmzdTMFNSKOQmYchgOv+4XBTUSy7h/Yo7VhDGPgmNWWqtXwPwWti27zpe+wDcFuXY/wDLR4QkpLqa7eAOHqSQNTZSmAIBCs1bb3Hc5Mm0EhcsYJLMpEkUJNN0ICuLx3i9dJ1On24t0pYWNizQmu5Tj4dJOoEAxxqX6K5dVni7uymCJSUUssJC20vWPCZPpnAeP87z3H8/LcRHH2XnIHNPaWmRBTM9nc/PPce5ZGTw/qRxgSCMXaTdnTAibNxIwcnOZqzR7abgmU45aWnA1q2MW1ZV0cqbNo3jTLs6rWl1GjfryZOMI65eDbz0EsXo85/n+JISCmV3NwWuqIji2N9PQQQowG43BfbkSe6bO5f7cnLsUl1NTXTD9vTwum1twAMP0EL98pdpiRrrMi3N3nNKCoXSJAGZeGpHB9fQlMWlBWHsImIpjAgm69Tr5bPLZTvqGHepz2fXiaytBa64gm5Qkw3b32+TeBoauM9kzwI87mc/ozDfeitw8cVc7Dk7m27fjz+mRVtcTNft7NlAeTnn4HKxRKSoiOeaO9fO6/hxCp8pMcnO5mPtWl5/0iSWsTjHTJ9OsTYNFEwrPeNS7u7m8YIgjE3GaumIkOSUlQG7d1PoPB5b0uF2M9M0N5cPE8srKwMOHKAVl5rK8ab5ud8P1NczzrlxI/Dss8Avf0lBTE1lWclPfsLzPPkkLctJkyhQzc0cM3Eizz1nDpuz5+ayaUF1NYXN4+EcOjqAzz6jAHq9nPOkSXSl1tdzvvffD6xbF5o16/XyWu++y3GmA5Hbbdvy1YfneguCMGYQy1IYEaqq6P5sa6NQ9fXZ9SM9Hm5ftSp0fE0NXxcUMCaZns7aRrPtyiuZZLNmDTvzmEzZd94B7rqLApaXx+v6/Ty+oIAu19ZWite77/LaCxZQkGtrgWPH6Hbt6AD+5m8Ys3S7KZBTpjBu2tZG9y0QatHeeCPPk5dHETXzTkujgAKci3HJSmasIIxNxLIUhoUz41VrYMkS9nmtqAB++EO7zxTxu92ML65aFZrwUlFBV2ZrK8suMjPp1jx1ikJ1wQUUu127bBcewLa527uXz7fcQuvx/fe5f+JEnk9rPufkMEt1715eo72d4j1rFl29//zPFMjWVm7PyGD8sa0NeOSR0HuvqBiY5bpkCWOep0/bbkL9/bzv+fNp+RrXsyAIYwcRS+Gsqa4G/vZv6W4FKC5vvMHyjscfpyDEs96j6epz9CgbpJvyEq+XFp/bbRNxTMccU4upVGij9dZWxigvv5zlIk1NFNqnnqJFOmUKRXfbNlqSHo+ti2xpofWXmcnjP/yQ5ywsZC3oa6/R+nU2GgjvSHTddbz/ffu4TWte48or6UZubpZSEkEYi4gbVjhr1q+nUJpWcAAttaNHKQjxYLr6NDdzPcq+PgpWXx/dp6mpFBmTiONcuDm848/p08CWLcxaLSxkws+iRRTKigqKWWsrx+bkWIvT66UFa0pbamoY91y2DPjLv2TN5eTJoV2HqqtD5272bdoE/NVfUTSzshjvnD8fmDnTXre2djifuiAIiUDEUhgy1dUs33jxRYoKYC08n491iC+/HF98ztnVp6QEWLqUQhcIsAH6T3/KJJrmZrozjWgCoU3XAV6/tJQCXl1t44jGiquq4nmam4ELL+T47m6e37h/3W5at93dFM23347cdWjjxugdiTZv5udSXk6r1uVimUxjI69TVnau/iUEQRgtxA0rDAlnf1eT6dnWFtpU3CyYHE98rraWVpmhqIirfdTVUZABlnwYV+dNN1GkW1tDhTI3lxZifT0tudmz7fEGk8lqznXNNSwv6e6m8Hq9tGQnTWKSjt/P6+TkhJ7HaR06597QwFhoTQ0t0UCA8zEu41OnGC/94Q+H8okLgpAMiFgKQ8JpTU2dSivS57MlH319tLJaWrjKx7p1zByNRllZ6CofwEDrKzyRZtky4LHHWA/Z0UFLUClapi4XBaqri2LpXN3EnMcZb1y3jq5b0wghNdVanP39FMaDB3ktI5yTJ1OMATv3hgYmFRm3sMnWNYtS9/dbt68gCGMPccMKQ6K21lpaixfTZTphgnWJ9vfTFZqVRevwP/8T+Na3ortkna7R/n772tl8IJwVK9i9Z8oUNgtITaVwm56vx45xVRJnLDG8obmxkNvaaAUby1QpHhsIsGlBRQWTgVpaeJ8tLXy/YEHo3PfupeCeOEGh7u62q6EUFHCeubmMXcYbzxUEIXkQsRSGhDNJpqiIVt6MGXRd5uezbCM3l0IZCFBgPvoo+uobxjWal8djwuOM0aioAJYvZyLNpEm0KLW2sccFCyLHGQ3GQj5+nC7XqVMpugDfHz1Kd6pSzIzNzWXyUm4u39fUhM7diHNhIctN+vspmIGAbbDg8UiCjyCMVUQshSERbgmmpTFZZu1au/zVyZM24ae4mM0IwsXKSUUFXabPPMPneMsqqqookgsWWIuyoYHXrqvja0O4SBkLubXVrkRimiiYdnXz59O1um+fdcHOnUvr0JzLzH3GDLpn8/P5A2HCBJ6jt5efE8BzvPFGaD9ZQRDGBgkVS6XUl5RS+5VSB5VSD0bY71FKvRDc/4FSalrY/jKlVIdS6v7RmvP5TjRLcMUKLomllC30nzqVSUA5OefeojL1jW1tjFGmpFCgFi6k+H3yCfDrX9sa0PA4qLGQc3JYcvLZZ3ykplLg58zhvu5u3md2Nl9v3QocOjQwozU311qT+fn8geByWdey308L2MxXOvkIwtgiYQk+SikXgJ8CuB5AHYDtSqlNWmvnIs7fANCstZ6llPoagB8B+Kpj//8A8PpozVkgkTrXAMDKlXS3AjYD1OejgJ3LkglnRm5FBc+9ZQstu0OHGDPs6qKAbt5MAXS7gW98w56jqornSE+niPf22mW3amtpQW7fzntoaWGST3Ex39fUAN/5TuicKivpfj1+nHFLl4vvOzvtkmFnztAKbmriUl9f/nJogwNBEJKXRFqWlwI4qLU+rLXuAfA8gOVhY5YDeDb4egOAZUox31Ap9WUARwDsHqX5CjEwVucll1AYALZ/83hiJ+0MhUj1jb29bIXX1ESBVorZqG1tbFIQHgc1czVLdZmetW43z7l7N7sHmfMDjEu6XGzLFy5wVVU89uKLKapTp9KS9HqZ4FNYyPls304BBSInHgmCkJwksnSkFMAxx/s6AJdFG6O17lNKtQLIV0r5AHwHtEoHdcEqpe4BcA8AlEk1+DklvNWbsZLWrw/dV1JCqy6WBRXtfOGE12YCFKRDh0LXmQwE+OjoiHyeigrunzaNY9rbKbQpKbxGZqbdn5Vl60grKyOfy9RwHj9Ot21Hh22inpJiE56am9mwwIiwtL8ThORnrNZZrgbwP7XWHcrZ7ywCWuunATwNAIsWLZIqt3OE0xXqLM8wFlw0V+1g53voIVp6fj8tux07bI9ZJ5FqM6dMoRilpPBh2uYBjDlWV0eeT3c3mwV4PBREl4ui2dfHxKX2dptpqzUt5mgWsrnnxka6hbu7eYxpqA7QHdvby5goINmxgjBWSKRY1gOY6ng/Jbgt0pg6pZQbQA6A06AFukIp9WMAuQD6lVI+rbUsr3uOiGblme0vv0zrbeHCUFdlNCspltW4bh0tw+xsCojPx/eRmhqYeCNgM1pdLlqXLS0U20CAFibA52jdhLKyKG6G1FS7luUXvsBEnb17eY20NCYxxfoRYBoPmFVWTFYsQPdwZiZdtYC0vxOEsUIiY5bbAZQrpaYrpdIAfA3AprAxmwDcFXy9AsDbmlyptZ6mtZ4GYA2Ax0Uozx2RGoQ/+SSwYYPdbiyt99+3JRrRrKQNG4A77mCbukOHuMhzeKxu2za6KNPTKSjp6Xy/bdvA80XLyF2+nCKUlmZjj1pTrPbvj7wCypQpjC0CFFnANjd49VVg505amPPm2aYDq1fbRuqrVwN33223ARTYq66yrmCz4HVWFi1Yr5fnef11XqOxUeKWgpDsJMyyDMYgVwF4A4ALwDNa691Kqe8D2KG13gTg3wA8p5Q6COAMKKjCCFJdDdx3HxNlCgvpLjRW0Nq1TGDJy2OpRHc3//Dv28cxkayk6mq2plOKlp/Px2zSBQtCrdBo3vRo2yO5ee+9l+L5wQe0ELu7KU7TplE033xzoDu2spKWXn29tVBPnKDgmS5En33GHwKLFnHdy+Zm4OGHec6ZMwe6oY2b+KKL6OLt6OC5tea1/H4u9zV5Mps6xNtHVxCExJHQOkut9Wta69la65la6x8Gt303KJTQWvu01rdprWdprS/VWh+OcI7VWusnR3vu4xFjUTY10drq7raWY04OBcW0ups7l8KnNV2f0drUbdxIF2ROjrUYvV6KkNMKXbKE8UET5+vu5vslS+Kf+8aNdrmwQICiXl5ue8fm5w9sjGAaG1x8MXDzzRSyjg4eW1xsY6FpaewHa1zOTU2Mr0bqEmQaN0yeTNFtb+c5XS7ObeJEzjMQAP70JzZz7+uTNniCkMxIBx/hz5iSjMJCioYRNtPBprQ0tNXdFVdYyy9am7raWmtRGrxeCo3TCl25klYbYK8xaxa3x8Lp5m1upss0ELDX2b2b7eumTh3oJg536Z48aV2zRtzNiiFO/H7rtjUYN7Q5p2lq0NNDkfR6Oa69nY/aWiYM7d7NetB33419r4IgJIaxmg0rjACmJGPuXHaqAejGbGqiCK1axcWNAQqBaXU3mPuwrIxCaTrpeL0Uw9TUUCu0ooJLV8VTOuIkkpu3pYVu4hMnKFKZmXSp7t4NXH31wHM4XboLF9oesYbUVJugY/B4Bp7H6YauqOAPiqoq4L33mLikFMVz/36bdZuayuYFXV3Anj3Rs3YFQUgsIpbCnzGxNmM17t1rY5dGEJ1rS5aVxa6fNJmr8+fTjXvyJAXi0UcHHhdPuUl4Vm1jI4WsoMBaggDdmmlp7Nnq9dplxGJUGmHJEpZ9KGWPM2Ulzc02+7awkO5i57bm5tAuQebHh7EwjaWute0w5HZzriZLVmouBSE5EbEU/oyzJKOggGLT3BxqOQ61ftJZrO/1csHls23xFqm28803Oc+DB63rNT+f4jRzJgXK9ICtrBzoOg1n5Uq6Y5uaeJzHw2Sku+5iYpIRabOA82A/HMyPD6elrjWFt7WVsVS/n3OeOJEPqbkUhORExFL4M05hi9dyjPe88ZwjVi2ms80dwGelWI4CUDT7+21W64UX2kWaAQpXSUnsuUZzB69YEXl8NMyPj7w8Wqy7djFGecUVTOrJyLA1pT4fY8JScykIyYmIpRDCUC3Hc0WsjkDAwDZ3DQ1cMQSwq4K0ttKaLC9njHLXLlrJpaWhzdQHE+Zz9RmE//i48UZ7nQ0bGGs9eZLzmzmT8ztX/XMFQTi3iFgKSUEkq9FsN8Jl3Jqmq86+fYz9ZWUx3udy8ZGdzezVyy6zGa4tLTZOGo8wnyuiCe+KFQPjv7ICiSAkLyKWwqgSzaKL1Bw9vCNQVRUzcvfsofu1s5Nu10CApR5ZWYwJ7tnDxJ7ycj4ACmJNDUUqHmEeDRJlxQuCMHRELIURxSmOHg9w7NjgXW+czdEjdQRqaKD1CNi+q1qzTGTWLB4TCAxcGcQpvPEIsyAIghMRS2HECHd3vvEGyzemTBnYfD1Sc/TwUgzToWfOHFqWHR1sRWdWGTFlKdddZxsAGJzCG68wR7unwVynsRrQi8tVEMYm0sFHGDHCF2nu6WG5xL59dkx415vw5uhOQQnvBpSVxfcdHRTLwkLGJR95xLbf6+8f2IrPtKOLtj8a0RrMmybo0fb/5CexG8kLgpDciGUpjBjh7s6cHCbkmHZ2wMCuN4NZW2VlrEusqeH7vj66ZTMygFtvpTW5aRNF9v77ucrIK6/QTbtkCfDpp9a6y8igeNfVxV8iEyvWGWn/yZPAj3/MkpXBGskLgpDcJNSyVEp9SSm1Xyl1UCn1YIT9HqXUC8H9HyilpgW3X6+U+lAp9Unw+drRnrsQm7KyUGGcO5ct3kw9ZLwWncE0PV+wgMJYW8v3115LMXI2MweYAHTVVcAtt9D9+8ADtOqmTGH8tLMT+Iu/4Ng1a0KX2YpEba1tJG/IyWF5yurVwC9+wddmyTKAXYv8fttIPhBgfPXll4GnngJuu42NEMKX+RIEIblImFgqpVwAfgrgRgDzAHxdKTUvbNg3ADRrrWcB+J8AfhTcfgrAzVrri8D1Lp8bnVkLQyHc3ZmWxiSchQuju1oHw7hqy8ttktCtt7JxusG4dcNdwMePs6Rk/372an3vPeCjj4CHHoruVg0nXPwBulWPHLGrjLS2hq7xefIk+9T6fHQXHznCHwx9ffxMNm0CnnuOTQrEPSsIyUsi3bCXAjholt1SSj0PYDmAPY4xywGsDr7eAGCtUkpprT9yjNkNIF0p5dFax2hmJowmkToC/fCHw3M9Ol21q1dTpJwcOkRr7ve/p3jNm8det62tFM3Dhym22dlskdfRQcvPJBydOsX1PHNzbUP2ykoKf6QkpJoa9r3Ny+O1tm6lBbl3L63X1FRur6/nuXt7uV9r23rP1I3W1XHNTHHPCkLykUixLAVwzPG+DsBl0cYEF4tuBZAPWpaGWwHsjCaUSql7ANwDAGXSS2zUGSwOOdwM0XDxOnSIVt2SJRSj1laK1xVXcP++fezuY5qtd3Vx3EsvMcO2sJBjOjvZli4lhc8ZGbRAp06lO7e21oro9Om0lhsaeKzfTyvy9Gn2wX30UVqPubnA229zv9vNRKf2drplleK1lAK2bx+46okgCIlnTGfDKqXmg67Zv4k2Rmv9tNZ6kdZ6UUFBwehNThiUWJml8RCeQVtfT6GcPZvxUa0pQHv20Mrs7ORYrSlm3d08T0cHW+O9+qqt00xPp6idPs2Sl23bgD/8gde8+GJaplVVFMyDBynS3d20YnNz2RS9qopNEG65hW7gnh66oktKbH0owDm63bRE+/po0QqCkFwkUizrAUx1vJ8S3BZxjFLKDSAHwOng+ykAfg3gTq31oRGfrXBOCY8phifnxEtFBd2xzzzDrj1mAeniYuDyy2lRHj9OAb3xRrbFa2ujVVdQQEtPa4pjXx+X/NKar48ds+5Sl8suMeaca1UVhda5pJfWNtu1upqW5cUXM5ln0iRaoW1tFM9AgPNNT+exqakUW0EQkotEumG3AyhXSk0HRfFrAP5b2JhNYALP+wBWAHhba62VUrkAXgXwoNb6j6M4Z+EcEauLztm4aMObDRQX01q75hqbaWqaJLz7LoXPuGV7evjc30/ha2igtWdwufh+715aj8760OnTed22Nm6/5BIKcXiiUWOjvVYgwB8JKSk8t89Hq/OSSwZ2HxIEIfEkTCyDMchVAN4A4ALwjNZ6t1Lq+wB2aK03Afg3AM8ppQ4COAMKKgCsAjALwHeVUt8Nbvui1rppdO9COFsG66IT3vnnwAEW9U+fbpNtIglnrC5AzoQjpShaM2cyfgjQ5drYSPFqbaV4AbYm84ILbDassz60snLgvTQ3c7/5UdDYyPhpayuty95eCmdnJy3R7GzGVmXlEUFIThLalEBr/RqA18K2fdfx2gfgtgjH/QDAD0Z8gsI5xWktdnSwJtHtHriEVrg1VlNDcTNlKOErhDjPm5lJt2l4s4FwS/Xhh7lEViBAsfL5aOUtXkwBc7s5x6wsimlbG8fm5Nh5GBEeTKQ3buTrvXvppg0EKMaZmbRQAwG+Nq5iaYMnCMmJdPARhkW87lKntdjWBmzZQlHLymJijHMJrTVrrIvWiIzXy+PCu+ZUV1P4mpp4Po+HWa3OEpVIS3Jt2gTceSfw85/bNSVnzaKQ/dM/8ThzTE4Ok3h277Yi7uz4E2vR7Cef5PwmTeL5/X4m+Zh7qqy0rmJBEJITEUvhrBnKupAbNzJp5o9/pKXoctG9mZJCF+T8+XYJLaeLtrWV+30+2z3HGdtcv55Clp3N7T4f369fz4e5dqQ2de3tbAgQTeydAjh7NvDgg9GtvmglMkZI77uPgllSQoF0uej2bW5mFu511/HzFKtSEJITEUvhrBnKupC7drEhQEMDRTMQYBwwI4NCW19vVwpxujWzsymYWrPzDxAa2/z1r3mu7m5abgDF6MUX6easqho8mWiwOtBztd5kRQVb25kfFj4f45dHjwLXCgMJAAAgAElEQVTTpgEXXshuQlVVwPXXs/2diKYgJBdjus5SSCzReqVGWheypYUi0d5uC/D7+xkXPHCAtZAmocZZP2nqIufPp3vVxAsXLKD4BAIst+jtZSu5I0coxm63tXTT0ga2qYt3Sa6hUl1Nd2p4r1fnPfX20l18wQX8wfCHP7BBwsSJwM6d0vJOEJIRsSyFuAmPT3o8FJ141oXMzWX80QhlTw/FMiWFMbz0dFqXxhXptOrCr+tMArrgAq5p6fHYc6an02Iz8+rpsW3xoq2Vea4+n8Hc0uZRXU0rcuJEumZ7e2l1e70U/lmzpOWdICQbIpZCXEQSgmPHKHwzZsQWocpK4JNPOL6lJbTFWyDABJfJkyOLRCR3qEkCWryYbtfOTlqUAEVo8WK+zslhZmx4As6VV/L9mjXnbjHmeN3SGzcC+fl83dFh1+fs6aFgfvIJ70cQhORBxFKIi0hCMHMmrcK8vMhZoE6qqhhfnDCBZSJ799K9OmECj8vPZ1zz+HE73ly3tpaWo9YUFKdVW1QELFvG8+3eTZfrsmXcDlhLN9xSdWbQ7t4NfPjh0Ju8h1u8u3YNPD6SW7q2lj8etm2zzdRTUij2JSXS8k4QkhERSyEuoiXJ1NXFV/JQUcHSkMceYzJOQYEtCZk+nb1VlaJ12dxMMdOagpyaylITgOtThlu1BQUUyawsbjPrZUazdJ0ZtC4X72H/fsYZn3kmPsGMZGkfOcKayfJyOy6SW9pk+15xBa9trMjcXM6np0da3glCsiFiKcTFYB13nAxWd7liBUswNm6kFXbkCBN36uvtslXz5vEaTcFeTFOmsJF5ZydF8MMPgZtuimzVPv44jzHnN0tsmX6zZh7bttGiDQR4bVPGcvRoaIxxsHuJZGmb8pdJkwZ3S5ts37w8JiqdOMGxOTmMt86cyc9JEITkQcQy0bS309S54w7gpz8F/vQnVsoXFyd6ZiHEaiUHxFd3GSlxZ/v20LUnAQphVxctzs5OikggYMtPCgsHt2oPH2byj+m445yHWe3j1CkKpcmmdblCm7kPdi+RLO1ZszjnaG7pSJ2G8vIo6osXUyTN5yot7wQhuYgplkqpbAAF4St7KKUqtNaS4D4Uvv514PnnI+/7znfs64suonJceSUr4bOyRmd+gxCrSw0wsE3d9u201n73O+ArXxlYP+gUznCr1eOxK3xkZlLMAIrmvn3cH630I1aizZIlbKRuRLi3l8I1bZqNMcY6RzRLu7IysoCH/5Awovj979vzxor7CoKQOAats1RK3Q5gH4D/UkrtVkotduz+95Gc2LgkmlCGc+oUTaoXXgDeeWdk5xQn8bS1M3WXjY3A5s0s6UhNpUW4ZQvjkJHqB6uqbP1kfz+fCwuZ9KI1k3/8fj6KiuiiHcz6ilX/ee+9tOLS0phJ29ZmxfjgQdsAfbBzRJrzYHMabEky5zJjq1eLUApCMhKrKcFDAD6nta4E8NfgCiBfCe5Tw724UupLSqn9SqmDSqkHI+z3KKVeCO7/QCk1zbHvH4Pb9yulbhjuXEacr399aON7eymaL77I+oIEMthCzc4i/MOHgUOHmJna0kJxa2nhrfT2suD+zjtDi/WBgYs45+UxM/Wmm2zzgmnT+PD5KKSRWuoZyspouTU0UKRffplxT2fTg8cfZ4JNIMB45cyZzELdto1xRHMOJ84YbaQ5DzanoTRwEIbA17/O/ySxHgUFwBe+ADzySMK/T8LYJJYb1qW1PgEAWus/KaWuAfAbpdRUAHo4F1ZKuQD8FMD1AOoAbFdKbdJa73EM+waAZq31LKXU1wD8CMBXlVLzwOW65gOYDOAtpdRsrXVgOHMaUeK1Kp10dQEff0zr8uabz/2cYmCsyZdfptAsXGitIoCh1s5O61r0+WgQ+/0UFrfb/q2qq6PrtKAgdizTsHJlaDNz47ocTJQAWncPPUThnjCB1m1b28CmB/Pn877q63nu3FybpBNPjHYo7fDiTZAShshQvDXNzXR7XHZZQr5PwtgmlmXZrpSaad4EhXMpgOWgUA2HSwEc1Fof1lr3AHg+eF4nywE8G3y9AcAypZQKbn9ea+3XWh8BcDB4vuRkqFaloaeHNRIJsC6d1qTWfLz/Pq01gAKybVuoa3H2bMYDOzvtwsbZ2Tw2JYUWYm5uqAtyMIZqvTmPmzqV1+7tpeW4dCnLTJzXrK2lRbl0KbB8OZ9nzrQ9Y8/m2tEYqttWiIOhfq8CAQbCk8BbI4w9YlmWKxHmbtVatyulvgTg9mFeuxTAMcf7OgCXRRsTXCy6FUB+cPu2sGNLI11EKXUPgHsAoCxRP+PPxqo0dHcnxLp0xthyczkNr5fJNcXFtrl5uGtx1iyKlMdDMUhJoeYbwZwzh+PidUGebTNzvx+44QZe09DfH3pNp7XX2Ej3cVMT3bxOC/RcEE+ClDBEzuZ71dk57O+TM36flkbPiVkeTmvW7O7bx/9bvb38fwfQ05KZye+Rz0fHkYmVCyOH1wv84AfAP/zD8M4TSyw7ARSBlpuTSxEqVkmL1vppAE8DwKJFi4blOk4Ivb38Nbxr16iKpbM0Yu5crpLh8TAGaayiJUsi94adMYNdevbvp1WWksJjp0+3FTHnygUZLfEoHrencbWeOsUWcykp/IM2ebJ1EwPxrdcZz5wiiW+864EKYZytt6avjx/2iy+yx+IQM82dWc2pqcyqBhjnfv99uvubmymeRiSdl25tHRgLF0YWn89+l4cjmLHcsGsAtEXY3hbcNxzqAUx1vJ8S3BZxjFLKDSAHwOk4j00ejB/TPK69lkV9kTBBwUce4dhAgFXrjz46qlN2JrgUFTEZRgV9DMYlee+9kV2Lq1bx9j7/eeCee+jizMmhm/ZcuiAHSzyqqmLC0euvAy+9xOfDh0Ovaay9+nr+IcvN5X3Ons17XLeOsc/XX2dy0uuv8/1gK4IMNqfhjBXCGI63xmldDhGnx2X/fnpRsrNZJpWdzbJpsRaTk//9v4d3fCyxLNJafxK+Mbht2vAuje0AypVS05VSaWDCzqawMZsA3BV8vQLA21prHdz+tWC27HQA5QD+NMz5jB6bN/OvsxHOmTPtY/p0dgLfujWhUwyPsaWlcd3Fn//cljdEi+utWBG6vbwc+PGPKULnIv5nGKwcA7DNBwzh7wHOYcYMznnpUmv55uQAb7/NJCHzHuD7desGnsdkBd91F/+I+v2R5zSU+QsjRF+f9dYMEWdWc2urbdnY1sZnE3KI9H9NSCxnzgzv+Fhu2ME6VKYP58LBGOQqAG8AcAF4Rmu9Wyn1fQA7tNabAPwbWK5yEMAZUFARHPcigD0A+gB8O6kzYQdj8+ZEzyAizhjbYK3josX1nMtRbdwIvPYardW/+7tz52YcbFHndev499D0WZ0zh67gSKuaRHPZtrVxX3o680FOnaJR8tJLtKqdnXmMa86ZDHX55RTfaPHZweYvxCBBauT8v5KTw1g+QKvS5+OPykDAtm8UkoeJE4d3fCzLcodS6v8N36iU+iaAD4d3aUBr/ZrWerbWeqbW+ofBbd8NCiW01j6t9W1a61la60u11ocdx/4weNyFWuvXhzsXYSAVFbQws7OBiy/m+6G4CofrZtywgdZeeTmfN2wIPffhw9y2ZYvN0m1t5R+st97iH6vsbP5Be/99/jEzQuSsD21spMUY7k42zcw7OmgR9/baJgvO+whPhlLKJkOZOUWKz5aV8bqmFnTLFr6XcpLkxelxufBC29TCLBVnSpWE5ONv/3Z4x8eyLP8OwK+VUv8dVhwXAUgD8JWoRwnjhnjXaDQ4E1YOH2aiT7zHOtmwAXjgAYrdhAnMVP3rv2ZTo69+Fdi0iec+c4ZW79atTLIwGYdmvUilaBkCtJBvvDFy6zmT0VhXZzNV16+ngLW12RBzT49dWNrcRzzJUJHW+FywgG5tE/dqaeG5viLfrKQlPKv56qvt/52lSyUbNhkZlWxYrXUjgCuCzQgWBDe/qrV+e3iXFcYKQ3EVhovQtm0Us+xs2yA9XjfjE0/Q5dnayj9EGRl8bN0KfPopmwfMns1zm5KP48eBp57igs5mvUiAXxatOZeqqsg/AGbM4LOzr+vKlRTPbdtspqxZWNp5H+Fu3NRUrqhiltuKFp+tqaF1sncvy2knTKDY1tQwhiokJ+eypEgYOwwqlkopL4BvAZgF4BMA/6a17huNiQnJwVA6z4SLUGEhraW9ewcuxjwY1dUUxKwsCqXWFM4JE/iLvLeXGayzZ/O8RUX89V5XF1o2csUVvLZxzV5/PfevWRPfD4CKCibsfPIJ7yM9ne7ooiKePy2N4mqWGysttaUyJSW0HM1alZHYtYv3UVTEFVJ8Pr4/i7wTQRBGmFgxy2dBt+snAG4E8OSIz0hIKobSeSa8/+mcOTymqWloJSPr1llLsKuLbq6UFKblT5jAlnknT4Ye4xRhM+e0NC4WfdVVtOBWruT+WH1fDdXVdPd+/vPcl5tLF9unnzK2WF/P61RUUBh37uQcc3N5vePHgR07gPvuixynbWnhfaWnW3dxSgq3C4KQXMQSy3la6/9Ha/0vYOnGVaMwJyGJGErbt3ARKi6miJi1J6Md60y2WbkSePVVHgNQZLu7aU329dEFOmUKXZ3RBDx8zsePA3v2ALfeyrjShAnx/QBYv57iuGcPhdft5hyOH2c7PeO6NW39AgHOtaODZXzd3VwIuqkpcmJTbq69P635bFoCCoKQXMRK8Plz+DlY6jHC0xGSkWgxmvDuMwsW0BIDbPNxt5txxGgxHhPn7OujpbZnD12XJSUUo88+o4j4/Yw9fvghxe7OO2nFxWodd+QI3ZqFhTxnSwvX2P72t6MfX11NofzFL+gKLilh/NHn4xKjZv3LggKOb2xkLNUsIn3iBPdnZ/O5sDA0IchQWcmED9PIPSeH7QLLy8/6n0oQhBEillherJQyHXwUgPTgewVAa62zR3R2QtKyYQPw2GMUh4ICisLhw8AttzBBJd7+pxs3Uih376YYGrdkfT0tyAkTKLidnbTuOjvp3v344+gWrjPR6MgRHt/SwnMZq+2VV5jpGu3YnTs5r4YGPtLSKH5bt9JCBWwsd+9ezj0/n1ZkTw+zYU+coGW5cGHkmKhpt3fxxaErm0hzdUFIPmJlw0bpySaMN+LtUWqsrhdeoICUltLiqqmhZVlTE5pRGovaWgqj10uRTE+36fanTtFaMwtAl5VRgHbuZELMunXAz342cO6NjTbRqL2d1lsgwPNlZVH06qM0RzTibSxag1mbs7OT9zl7tl3Cq6WFn0VKCj+z7dt5XbebSUbFxRTB8JioNFcXhLFDrJilcB6wYQNwxx3sLX3oEHDgQOQYm9PqSkmha7K+nkLk9TI+ONTuM2VlTNbxevl+0iQKk0l2mT6dgpmfz/MDtkvOW29x7uGND958kwIO0Jr0+ylcZltbG0U+ErW1tl1deNShv5/lKzU1oXFRs2bn7Nm0LKdOpUBnZtqEoEgWozRRF4Sxg4jleU51Nd2pZjF5YyUGAgN7lJrSkJYWWn+nTjFjdc8eWnPmj/5QqKqiGJolv1wu1jLm59vlj2bMCG0MkJ5Oq6+7m03bw3ux5ufb8ovFi+kW7eri+VpaeK5VqyLPp6yM92GW9jKZuEY4J060PwgqKmhFP/sss19ra+l+zcjgHKZOtQlB4S5jaaIuCGMLEcvznPXr+cf8xAng6NHBrcTaWoqpyXjt7eWju5vHnz5NF+VQqKjggipaWwtz8WLG+daupQjNns1r9PXxkZpqs2tN42rnwtSVlZxLczOL/JcsseKXm8um7tGK/quqeA2324qzEXG3m8dHc6f29PCRng4sWwbcdBOvY3rq3n03xdVYlNJEXRDGDrESfIRxTHU1XZZpaXzf28tOMlOmULiuuSZ0fFkZl6kqKAAOHrTWlllJbMIE9oUPF6JY7sYVKyiIkcaY7aaG0TQmmDKFFlx2dmgv1uJivr7uOopPbS1w2WXAj34Un4uzooLHvvEGRROgUJprFBbyB8Hq1QPnunz5wAYOhw4xyeiCC0ItyLa2gfORJuqCkLwkRCyVUhMBvAAu8/UZgNu11s0Rxt0F4JHg2x9orZ9VSmUA+BWAmQACAF7RWj84GvMeb2zcSHdhZyddqi4XH3V1FJ2qqlCh83iY+OJ2U1jNqgqm32VpqW0xZwhvgWfEItwtGWv1EpM5mpfHBXfT0iiaixczJujsxXroEF2g0cQ5lnh/9au2FKSz066mduml7OizaVPk+zFzBGx2a00NW/OF98etrY28cLY0UReE5CRRbtgHAWzWWpcD2Bx8H0JQUL8H4DIAlwL4nlLK/Gl5Ums9B8BCAJ9XSt04OtMeX9TW0mXpcjGxxtQJ9vTYtaadcbW2NhsrNK5Jk8VqFpwPX5boXLkbIyXUXH45hci5MLVJzElLixwLdMYKU1NpKVdVAd/6lh1TU8MG2RddRDfuF74A3HYbr1NTE/1+IjVwmD6dtZNOcnLomo23M5IgCIknUW7Y5QCWBl8/C2ALgO+EjbkBwJta6zMAoJR6E8CXtNa/BPAOAGite5RSOwGEdfoU4sH0UL38crowU1PZp3ThQrpGV68O7fVaX28L+/Pz2anG9G0tLqabNNxFuWvXQIvR5+OakEPNAg23Mj2e0IWpTRmGxxN9pRMj3j09tIK9XibtfPSRtRBraylws2fba/f3WxdpaiprNE0jAZPc45yjYfXqyL11KyttU3cpGxGE5CdRlmWR1vpE8HUDgKIIY0oBHHO8rwtu+zNKqVwAN4PWaUSUUvcopXYopXacDG8oep5jeqh6PKE9VO+9l/vDe722tlJMMzKsK7avjxZWRgatU58vNMPzyBHGNw0NDcB77/GaZ5sFOlgLvvA5A6GxQLPfNBIwtZ09PdZCHKx3bFoa59/dbdfKfO89G/eN9hlHsiBNNu0zz/BZhFIQkpcRsyyVUm8BKI6w62HnG621VkoNeU1xpZQbwC8BPOVcFDocrfXTAJ4GgEWLFsna5Q5iFcWHrziSk8Os164urulYUkKx6ukBLrmEY9LSQq0606igoIDHf/QR911yCZOIzPJa991n2+LFU38YLcYZa5UUs7+1lWIHUOBzcqzFO20aRX7+fFqYprOOWeMyEtE6QUrjAUEYH4yYWGqtr4u2TynVqJQq0VqfUEqVAGiKMKwe1lUL0NW6xfH+aQAHtNZrzsF0z1sGW5svPGGltJRt5goK7AK2xcUUlaIiioHpl2qYOZNuWpOZ2tNDCxZgEo3XG9ps/JZboifQxCMwCxaEtuGbMoWxVbP4srmntDRahUrxPsrKaCFmZ/M6mZkU+a4uukyNwPn9nP/+/dYNW1nJ7WfzGQuCMDZIVMxyE4C7ADwRfH45wpg3ADzuSOr5IoB/BACl1A8A5AD45shP9fwl3CoqL6cw9Pcz2Scnh/HNwkJrNRmrrrHRWo2FhaFux+Zmiq5xg3Z322bja9eyV2p4zHHdOgrzYNamWVJr/nzGV0+eZHz10UftWHNP69ezbCY/n3WYponBJZfQrVxeThEPXxDa3OPSpXZbczOtbOnIIwjjl0TFLJ8AcL1S6gCA64LvoZRapJT6VwAIJvY8BmB78PF9rfUZpdQU0JU7D8BOpdQupZSI5ggRHle7+moK5vLlFIziYuvmNPG5AweAP/6RQuV2A5Mn27ikGdPUxLhldzctuzlzKL719QNjjj4fW9vF6nZjkndmz2aN6O23c441NQPvaf16jr/xRpsBfNVVdpFqIHrz80gxyAULpCOPIIxnEiKWWuvTWutlWutyrfV1JuNVa71Da/1Nx7hntNazgo//G9xWp7VWWuu5WuvK4ONfE3Ef5yOxElbuv5+C19fH8ogrrqB4hZdXFBaytjM9ndm4RnRLSwcm1+zaRQswVvlJrOSecJw/BJYvt/1pDZHqHqMlFw1WUiIIwthHOvgIQyJWwkpFBXu5XnWVbTEHhIpWRQWTeUyTgZwcNhbYvZtlHFu2hCbXnD7N9nFOIolgrOSewYjUUMAk9UT6DMLdq2vW0KKMNUdBEMYmIpbCkImVsBKPaDlFd9cuNmP3eKzFumOHTa65/vqBpRmRRHAoghfpnoaTtTocoRYEIfkRsRznJCLpJF7RMqK7ciWzSzMybJZtXx+TZkzj8XjPNxzBG07W6nCEWhCE5Efp8P5k45hFixbpHTt2JHoao4azL6vzD3i8ZRjDvXa8Ir1wIbviZGTYbV1dTLwxdZmjLfpnc71Yx2zYwGzf+nrGZletir76STSUUh9qrRcN/Y5GhvPtOyWMP+L9TollOY5x9mUFBrZ+G0mGYqVpTXFsaqJV6fUyi7ajg8taGeFxlnCEcy7FNN7m7+EMds8bNgAPPMA6TtMy8IEHuG+ogikIwugjYjmOqa0NTTppaGDt4/HjfD9UQYkmSMMVqvJy4He/o0h6POwx297OlUNiiVV1dWjNZGUlxz/8MK23np6hW4bPP2+XAysuZjN1Zzbv2bB2LYUyN5fvzfPatSKWgjAWELEcxziTThoauECyUqx7HGpnnGjW1nA77gAs/s/MpEC1tXFtTLc7tAwDGChWZk779zOLFmBz9Nmz2Y+2qQm44YbQudbURBZ1c66+Ph6XmsrjAgH+wMjNpZv4bC1W04TeSXY2twuCkPwkqimBMAo4ayL37qVQag3Mm0eLa/9+4M476d7csIHPd99tk2qcRFtqa+3a4dcXnjjBJb6ys/lwu2lltrfbMZHKMJwriJiG6F4vsH07rcKeHjunvj62wYu2NJc51/HjPI/bzc/q+HF+bh0dzMg920YDpaX8IeCkrY3bBUFIfkQsxzHOAvrjxyk4V1zBfVu32rUnP/2U8bMDB6Kv8Vhby3jili3Ayy/z2eeL3HFnqPWFLS202srL6fLMzWVnn/p6XqehIXIZhmlCYJqgA6Ei63LZ+W7dyu09PcA777ChQGsr8MordNnu2mWToIqLaVF2d1NkzeLYgQAfZ9NoYNUqimNLC0tjWlr4ftWqoZ9LEITRR9yw4xxn0olxyW7ZYrvV5OZSSLOzmXl6+rRdI/KPf2Tizf33M5a4ZYu1/szSVEZghlNfmJsLnDljxamri6Lk8fD1u++yIfvjj4ceZ9zMc+dSDAH+APB62Rc2NZWCmZ0NHD1Ka/G993gtj8f2pT14kG7gCy7g/XR306VcU8N59PfTzetyAZ98QvEcKiYu6cyGfeQRiVcKwlhBxPI8wVkH2NJCMfT7WbbxwQd0Vx4+bGNzvb38o37RRbSktKb1ZmKKLhfPsWABBQsYen2hcX8ePcrzBwK8ZmYm45h9fZxHdjaTfcKTiTwe4NgxCqlphn7mDK3njz8OrdtMTeW9HT3Keaam8tyZmXTZmnlPnkyRTEnhPXq9fJSWUlz9fn5+8RAp8WnLlrP65xMEIcGIG/Y8wemSVYqPyy/nc3MzO+gEAnafUhSH+nr+sW9ooMAA1n1rBMfZK9XvpwCtWRM59mkwCTXNzcCll1p3Z3Y2hSklhYIJUKxOnAg9ZsoUirVSvGZvL5uib9wI/OpXjMvm5Fhxz89n4k53N12xvb3cPmkSrdfWVo49fpznbWnhfRpBzczksf39NpN1MMLnKo3VBWFsI5bleYRxyRors72dbsW0NAqDy0WRmDCBYlVaSnfmNdfQ6szIoOVlaGmhQEay9AoKBs+MDa8BXboU2LmTlp9pbedyUaRaW3mt9esH1o3OmMHnqiqec80aWnHFxXSr+v3MAp4wwQpraysFb+pUvj92zFquBw8yo/bzn+e9mXhpQwOzWWfOZLatk0gWZCJrXAVBOPckxLJUSk1USr2plDoQfM6LMu6u4JgDSqm7IuzfpJSqiXSsEJ3w1UEmT6YIZGdTNLu7KZSBAK3HqiqKS38/95kxnZ1ct9JYTzt3AocOUYBiZcaGrxBSVMQaSRNDbWvj3Hw+Xm/BAopYpGSiXbsGWnH19ZzLRx/xPurqaJ1Om0ZrMjWVPxY++4z35fXS8jTx2/p6WqdeL4W3uJjrbLrd/DwM0SzIXbsiJ0RJY3VBGJskyg37IIDNWutyAJuD70NQSk0E8D0AlwG4FMD3nKKqlKoC0DE60x1/mNVBVqygVXfVVbScZs2iBWlE0SycXFlJ66uxkdZoYyPPM3myLRvp6aEFt2+fvU60zNiystCluBoabMJQQQGtyiNHKHRXXEEx13rg8l3G6gwvX5kxg/NtbbW1jNOnc36ZmRQ9I5Ll5bze++9TULOzeVxREa+dk0MRNctxAbbM5r77OMfw0pm6Ot5Pd3doQpTHc67+BQVBGE0S5YZdDmBp8PWzALYA+E7YmBsAvGnWulRKvQngSwB+qZTKAvD3AO4B8OIozHdc4mxaUFxM9+LWrbToCgtDe5cuWAD8/Oe0/MrKaPkdOkQRNeTk2PifIVpm7IIFrHtsa7Mi6HIxochkqQJ8XVTEeS5ZwudTpyhGJuN14sTIFmddHe/L6w2NM2ZmMhb5l3/JRKDubl7HzLetzY4vKqJb+JprbD2qc96nT7OGtaeHy4qZa7e3hy5RZjiPWjELwrgiUZZlkdb6RPB1A4CiCGNKARxzvK8LbgOAxwD8BEBXrAsppe5RSu1QSu04efLkMKY8/nA2LThxglmg2dkUyIsvZmee6mo+1q7lH/qWFrorc3Pp0qytpZVp6iEPH6bYnjjBes1XX+V2Z2JLdTXPXVpKcW1ro2Dl5rJRQmGhdb+2tNg53nuv7cJz8iQt0PnzmQF76FDovRmRjuQ+7u/nIyeHZSc+H7d7PBQ40ywgfHHr6moKZXc3593dTavS7wd+/3veZ2Mj8MYbnF9GBve3tVGMr7qKoioIwthjxCxLpdRbAIoj7HrY+UZrrZVScf/eVkpVApiptf7/lFLTYo3XWj8N4GmAKyTEe53zAeeSVi+9RKG85BJaU4b16xmbbGqyCTE+H0Wmvx947TWKwR8hXUwAABdlSURBVIQJdG329VG4TLLLsmUUIZPoA9B12dREwSksZKbqgQMUH5O1esUVjIECtHzNUlsbN9JtnBcW5a6pYSwyUvlKRgbdqK2t3D9zpn1vXK179/K6ZWW0qJ1t8cy1V6+mRdrTw3tNTaUAd3ZSZLdv5z0BPG99PcfOmAHMmcPPIbzlnSAIY4MRE0ut9XXR9imlGpVSJVrrE0qpEgBNEYbVw7pqAWAK6K69HMAipdRn4PwLlVJbtNZLIQwZkyFrmq47XYc5Oexwc9VVFDWnu3LvXlqfpkbx+HFaUcXFtqlASgozUI2wOYXX62VyTUMDXZk5ObTKjDWZlgZceOHATNrw5vAA46xdXbxOpHUsn3ySc3UK6apVtG4BWqhpaXb5MoBi6aS6mj8ozpzhj4PsbIplerqN1X72GedSVkb3bl8fx5w4Eb2xgiAIY4NExSw3AbgLwBPB55cjjHkDwOOOpJ4vAvjHYAxzPQAELcvfiFAOH2f80tDaSvHKyaFl9P773O7xUPAOH2ZsrrSU79vbKQxGJPr7mexTXBwqvBkZFBaXi2Jq2tAVF9saT6c1Gc88KysjL+EVbUFogPN4911eb8kSK5TOhvEHDtAtbdy3WVk2qcjENT0e4AtfAHbvpvX929/amGtqKoV4yhRbniIIwtgjUWL5BIAXlVLfAHAUwO0AoJRaBOBbWutvaq3PKKUeA7A9eMz3TbKPcO5xdvhxWmBLltiEndRU1kEGAqxhNCJp1qI0JSMARbax0cbonMJr8HrtIs9paRTPSNZkPPMcrGNQ+DqTzhVUbrnFngMIrY9sbKSF2dbGuU2cyKShggLGJJubuT0/n+PS061Qms+hp4f3VlnJz0cQhLFJQsRSa30awLII23cA+Kbj/TMAnhnkPJ8BWDACUzzvGMwCe+ghxiFNCzjT9m7/flpLmzdT8IxAmCW2tKZAmCQZI7zt7bbO0YydNInnjmeB5UjzHMq6mtEaBqxbx1pOgFZjR4ftodvXR1EEOO/CQv5AMCUj7e1sSG9+HJhuSKYL0K5d7DAkCMLYRDr4CH8m3AIzTJ1Kt6lZvmrmTIpBdTWFQmv7SEmhwLhctBo7Oxm/W7WKpSkPPcQYpWlw3tNDi+1zn+P+eNyUkeZpSjp6e2n5+f3RF4COFPf0+Sj6+fk2a/bwYdZmulw872ef2dVNCgo4f3O/fX3c7ywNMS7lvj7es7OZgSAIYwvpDSvExO+niJSX8zFhAgXE76eFmZlJEU1JoZWYlma7AC1ebMtQAApvSQnFRym6czMyGO87WzExJR2mn+3Bg4yP/v73XDklvDdreEMEgJZffj6zgY271PTGNda0ca9qTWHs66NQmgbwBqV4jDNZ6rrrJF4pCGMZEUshJmVljNEZ62nfProc09MpnIAVkf5+jsvI4P65c0Pb3vn9wFe+Anz1qxReU+84ffrQxaS6mkk9d91Fq9fvZ0wR4Fx6eihixl1q5uCsLzW1lKdPM65oSknS05nM09PD+U2dyh8CPh9fX3ABmyN4vbx2l6PiV2sKJsBj8vJYIyoIwthF3LBCTKqqgP/8T9tM3eWiiLhcbHfX10ehysuj27Wnhxmis2czwxWwbe9MNmtRka3nDM9udRItDulM0tGa1uzRo7RyTf0jQNFzZuTW1kaOe15/vW3gbubW3Mx72baN5zOLUxcVMev3V7/ij4fGRoquE7OM2dSpPLdYlYIwthGxFGJSUUFX6jvvUDwyMig8vb18DViRLCy0a1E6GyaZjjpDyWZ1CqLTlWqEzliLubkUp1OnaPllZFixLC62Lldn671oGbLh87r/fiucTkH3eoFFi/iZ9Pfb+GR/P63J3Fw2I7jwQmDlyuH/GwiCkFjEDSvExYQJwK23sr3cxIlMcAEYH6yro5BMnsyMz8WLKRpNTQNbxjnX1TTH3XILxe/uu0PXwHQKYvgqJs5VS+bO5f7sbIpmVxfFbNIkvs/ODp1DJMLnZX4UrFlj2/g53bbNzRTDrKzQeK1plN7WxoW1Y2X3CoIwNhDLUogL4z5dupRux61baUX6/bQqGxqAa6+1rtWLLmK8sK5uYEcdp1UXbj0eOADccQdjmJ99Blx2Weg8wt25eXk2zviHP/B6ZsWR/fs51szTKVzR3LvhLt7CQlsj2tMTej9r1lAc09NtBm1fH4Vz6lTgZz8bjX8ZQRBGAxFLIS6c7tM9e+hy9Hq5GgfATjhHj7LTj1lB5KmnYltVkZoAKEUh9Hi4rNXVV9vYZ7g7N3wFkkceoeC+9RazW2+7jfM0TQeAgQL96acU6IkTKXonTtCNunChtWhnzuSz6RJUXU1r0+ejiGZlUcj9fgrmtdee049fEIQEI2IpxIUzKeb4cbpc586liDU00GV54ADFwrSOCxfKSNacs+Zx714Km9dL4czK4rVeeQW46SYresZKveWW0NrK0lLWdGZmcvmt8KShdes435dfZjLPwoV0Fe/eTdHbvZv31dREwXvpJQpgSQljj2ZdTiO2kyfTem5stC373G66qiVOKQjjCxFLIW6c7lPjAm1oYM9Ys8hxTw/fm7rC6mqK1Ntv0wosLaXL1CTrZGTQWszL43N2Nhd9PnGC1ilAIfvlL5ld++CDdg41NQNXIGlupjV6882c2759PG9KCkWwqso2UHj/fQptaysfJjknNdXWTZ4+DRw7xjrM66/ntnXr6OI1ZSWpqbRsXS6WxaxcKXFKQRhviFgKQ8bpkt27l0J58iQtxIkTKTyPPUa37L/8iy2tSEujCL7zDmsajx7lWI+HItrcTOu0s5OiZZoBAHZ5sMceA/7jP3j8rl0DRSknh0J48CAtRa+XArx/P63Fnh66WE1j9CNHaMH6/Txu714Kq99vG70rxdZ3b75Jt/OuXZxvfj7nl5rKz6S3lyurCIIw/pBsWGHIODNHjx+nAE2ZQvEw3WuOHmW7ufp6imQgYDvgtLQwxqk1xWzyZGvlOZuN9/RwfFoarcT6ehvPbG6m0B08aMc3NHDh5e5utq7r7ub5TVyxuJhiaBZ8Pn2ac2hv5/FpaRTQjg7b19X5aG1lEpHPxzhpZyeTe7xeCqgpSxEEYfyREMtSKTURwAsApgH4DMDtWuvmCOPuAvBI8O0PtNbPBrenAVgLrnfZD+BhrfV/jfjEhT/jdMm++CItSoBCc/QorTjALo4MUAxbWrjNtJILBLitsJCxwfZ2CmYgYMVUKZ7DlG34fBTqBQvoii0o4Lb33uN1li6lFdjaymNLSpghm5JCQe3o4LWbm21DgwkTWHLinLcpB/H5bN2m6VLU2ckY5ty53HbmjPR+FYTxTKIsywcBbNZalwPYHHwfQlBQvwfgMgCXAvieY23LhwE0aa1nA5gH4N1RmbUwgKoquiFNecWJE9xuag5dLopLT48VmkCAYtPZSXdmZyeFq6GBopuVZbvp+HxWbAEKXXOzTQDy+ymSL7/May5dSnEsK+NzSQm3LVpEV3FDAy1Sl4vjAc6pt9e2qTM9Xc2anEYo3W77SE2luLa18Rjp0iMI45tEieVyAM8GXz8L4MsRxtwA4E2t9Zmg1fkmgC8F990N4J8AQGvdr7U+NcLzFaJQUQE8+igFxfSPLS6mRVhQQFHp66PoGKEEKFamt2xmJo/3+YBLL+U4I7LONnLd3bQ809LYau7f/53WXUoKE4VMP1aAFp+zMYLHYxdiTkmxq4UoRaH0+Ti39HRaodnZdsUQgK9TUzlXgOfMzORi1tKlRxDGP4kSyyKtddAGQQOAoghjSgEcc7yvA1CqlAquT4/HlFI7lVK/UkpFOh4AoJS6Rym1Qym146Sz/5pwzlixAnjuOeD22xl/zMwErrySz6WlNv6XksKuOpmZ3Gbcm1lZtph/zhxmy5rx4fT20jpsbuY5MjNtopDfz5gkwEYFF11E967pFDRvHuc3bx7F0+Ox1qWxKv1+Pt9wAzBtms3IVYrjs7L4nJLCRKG8POnSIwjnAyMWs1RKvQWgOMKuh51vtNZaKaUjjIuGG8AUAFu11n+vlPp7AE8CuCPSYK310wCeBoBFixYN5TrCEDAxTJMpm5dHl+quXRSk7Gx25cnKYmceE7tsa6OAlpXRrVlXx3Z5XV1sYG7ct4B9NsKal0fh6u3luJYWa5FGaoywejXdxD4fH+3tPNaQlmbjqh98QKs0NZVWq7E+AZ73c58DfvpTEUlBOF8YMbHUWl8XbZ9SqlEpVaK1PqGUKgHQFGFYPZjAY5gCYAuA0wC6AGwMbv8VgAhtuIVEEL6ix403UmzMgs9a0/Xa2kqXZ3a2bQJw8cX2PCZD1Vil4bjdjF+2tNi4YkYGH9XV3J6by3mYeVVVATt2AIcO0YJ0il9qqrUqW1qYsON201Wbn09LNhCgOH/5y1xyS4RSEM4fEuWG3QTgruDruwC8HGHMGwC+qJTKCyb2fBHAG1prDeAVWCFdBmDPyE5XGAoVFbTinnmGz1dfzczV9HRakvn5dIX29zPOmZ5Oy62mhqLV3EwLs6/PZsWGC2ZPj20tFwjYWGhmJrdNnszzvPgiW9lt2MB5Pf445+MUSo/HnsOcLyXFJgH19QGzZlHM/+Iv2PNVhFIQzi8S1ZTgCQAvKqW+AeAogNsBQCm1CMC3tNbf1FqfUUo9BmB78Jjva63PBF9/B8BzSqk1AE4C+OvRnb4wFIxr9uKL7fJXW7YAy5ZxjcgtW2zscP9+JueYBaQ7OmySjxEwk1WrtRXTlBQmAJm1J40I5uTQWly1is0MSkp4LuNe7eqicDoTiUxikVJ23U6vl8JueuEKgnB+kRCx1FqfBi3C8O07AHzT8f4ZAM9EGHcUwFUjOUfh3BFpseXp09mcHLBt7szr7dttAlBmpnXJas1M25YWit2ZM9b66++nwBor1FiabW0UUbebcciuLh6rFLf39AxcuNnQ22sTeVpbbaceQRDOP6TdnTAqhC+2vHq17S+bk0PhAih+hw9TmLKzQ2sstWYzAbfbZqd6PDzWmahjxvr9trl5dzffp6ZyW29v9Hio8xymI5HWLJER96sgnJ9IuzshIVRV2bZ1F15IC7CtjaKUmkrL0O2mZWkaFKSk2LpI455taxsolIC1ME1DBGM9mrpPU7ISjhFiM/ayy4A772RpzIoVI/NZCIKQ/IhlKSSEcNfs0qUUsM2b2We2vZ1CaoTOGa80DQ2cVmcsjCVqRNPloshmZvIagYC9jolX/tM/Af/wDyNy+4IgjDFELIWEEe6aBax71u8HfvtbiqbpJQvYdnQmYxWIHnM0q4b09fF5wgRb32nqMSdNstt6enhcairw7W+LUAqCYBE3rJBUGPesx8MuOllZdI2mp1PwTK1jfz+zWQFud4f97EtJYSy0tJSZrB4PRTM3l6Jp2tXdeCPwhS/wXEVFwOWXM2v2Jz8Z/XsXBCF5EctSSCrC3bPXXAP86U+0/rxeuk57e+06lV1dtqmAKQEx4rpgAbNY09IohKY20+PhtgUL+P7SS4EnnpDkHUEQoiNiKSQd4e7Z6mouqrxtG7Nas7JoHZouO62tjF+ahuq5ubQ2T56kiD7+ODB7dmjpSlWViKMgCPEjYikkPRUVFMtwjIi++SZXCqmsZJZsTQ3rOCsrQ0VRxFEQhLNFxFIYsxgRra62VmN5OfCd74gwCoJwbhGxFMY8kbJqBUEQziWSDSsIgiAIMRCxFARBEIQYiFgKgiAIQgyUHqyT9DhDKXUSXBLMySQApxIwnVgk47xkTvExknO6QGtdMELnHjLynRo2Mqf4SPh36rwSy0gopXZorRcleh7hJOO8ZE7xkYxzGk2S9f6TcV4yp/hIhjmJG1YQBEEQYiBiKQiCIAgxELEEnk70BKKQjPOSOcVHMs5pNEnW+0/Gecmc4iPhczrvY5aCIAiCEAuxLAVBEAQhBiKWgiAIghCDcSuWSqmJSqk3lVIHgs95UcbdFRxzQCl1l2P7FqXUfqXUruCjMLjdo5R6QSl1UCn1gVJq2mjMSSmVoZR6VSm1Tym1Wyn1hGP8XymlTjrm+s045vKl4P0dVEo9GGF/1PtUSv1jcPt+pdQN8Z5zpOaklLpeKfWhUuqT4PO1jmMi/juOwpymKaW6Hdf9meOYzwXnelAp9ZRSSg1lTolCvlMx5yLfqZGfV+K+V1rrcfkA8GMADwZfPwjgRxHGTARwOPicF3ydF9y3BcCiCMfcC+BnwddfA/DCaMwJQAaAa4Jj0gD8HsCNwfd/BWDtEObhAnAIwIzguT4GMC+e+wQwLzjeA2B68DyueM45gnNaCGBy8PUCAPWOYyL+O47CnKYBqIly3j8BWAJAAXjd/Dsm+0O+U/KdGu53aix/r8atZQlgOYBng6+fBfDlCGNuAPCm1vqM1roZwJsAvjSE824AsGwIv2DOek5a6y6t9TsAoLXuAbATwJQ4rxvOpQAOaq0PB8/1fHBu0ebqvM/lAJ7XWvu11kcAHAyeL55zjsictNYfaa2PB7fvBpCulPIM4drnfE7RTqiUKgGQrbXepvkN/zki/z9IRuQ7FR35To3CvKKdcDS+V+NZLIu01ieCrxsAFEUYUwrgmON9XXCb4f8GTf1HHf9Qfz5Ga90HoBVA/ijOCUqpXAA3A9js2HyrUqpaKbVBKTU1xjxiXgPR7zPasfGcc6Tm5ORWADu11n7Htkj/jqMxp+lKqY+UUu8qpa50jK+Lcc5kRb5T0ZHv1OjNKyHfqzG9nqVS6i0AxRF2Pex8o7XWSqmh1sj8d611vVJqAoD/AnAH+GslkXOCUsoN4JcAntJaHw5ufgXAL7XWfqXU34C/yK6Ndo7xilJqPoAfAfiiY/NZ/TueA04AKNNan1ZKfQ7AS8H5JTXynZLvlJMk+04BCfxejWmx1FpfF22fUqpRKVWitT4RNNGbIgyrB7DU8X4K6I+H1ro++NyulPpP0HXw8+AxUwHUBb9kOQBOj8acgjwN4IDWeo3jmqcd+/8VjOMMhrkH5zXqo4wJv8/Bjo11zpGaE5RSUwD8GsCdWutD5oBB/h1HdE5BV5A/eO0PlVKHAMwOjne6+ob6OY0o8p368zXlOzUy36lhzSuh36tzGQBNpgeAf0Zo4P/HEcZMBHAEDPbnBV9PBH9ETAqOSQV95t8Kvv82QgPPL47GnIL7fgD+kksJO6bE8forALbFmIcbTHKYDhtgnx82JuJ9ApiP0GSEw2DAPuY5R3BOucHxVRHOGfHfcRTmVADAFXw9A/zimn/H8ESEv0j090W+U/KdGo3v1Fj+XiX8CzhSD9C/vRnAAQBvOT7QRQD+1THubjCgfhDAXwe3ZQL4EP9/e3ev4kQUBXD8fyo/HkAsrFTEQmRBX0AUREFB9BFEfAXRysbCSrC1F7awEbHxA9KJQlZ3EYVtxMJSEFlQ5FjMVQcU72Yjmbny/8FAmAmTk0wOh0ku58BLuj+3b/Uu0HZguTz/GbB3QTHtARJ4DUzLdrEcu1HiXAGeAAc3Ectp4C3dqrSrZd914GztfdL9/LUOvKG34uxP55zxmm0pJuAa8Ln3uUyBXX+7jguI6Xx5zSndwpEzvXMeBVbLOW9TOmmNfZvz+2tOmVNN55Xt7iRJqvifV8NKkvRPWCwlSaqwWEqSVGGxlCSpwmIpSVKFxVKbFhHfSour1YhYjoidZf/uiLgbEevRTSh4EBEHyrGHEfExIu4PG700PuZUOyyWmsVGZi5l5iHgC3C59IW8BzzNzH2ZeQS4wq8enTfp2mFJ+p051QiLpbZqAuwHjgFfM/PnXLnMXMnMSXn8CPg0TIhSU8ypEbNYamalV+Mp4BXdrLsXw0Yktc2cGj+LpWaxIyKmwHPgHXBn4Hik1plTjWh66ogWbiMzl/o7ImINuDBQPFLrzKlGeGepeT0GtkXEpR87IuJwbyirpNmYUyNksdRcsuvEfw44UZa5r9FNbPgAEBETuukBxyPifUScHC5aafzMqXFy6ogkSRXeWUqSVGGxlCSpwmIpSVKFxVKSpAqLpSRJFRZLSZIqLJaSJFV8B5RPMDhInaAzAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff3a4120e90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(7,3))\n",
    "\n",
    "ax[0].scatter(X_kpca[y==0, 0], X_kpca[y==0, 1],\n",
    "             color='red', marker='^', alpha=.5)\n",
    "ax[0].scatter(X_kpca[y==1, 0], X_kpca[y==1, 1],\n",
    "             color='blue', marker='o', alpha=.5)\n",
    "ax[1].scatter(X_kpca[y==0, 0], np.zeros((500, 1))+0.02,\n",
    "             color='red', marker='^', alpha=.5)\n",
    "ax[1].scatter(X_kpca[y==1, 0], np.zeros((500, 1))-0.02,\n",
    "             color='blue', marker='o', alpha=.5)\n",
    "\n",
    "ax[0].set_xlabel('PC1')\n",
    "ax[0].set_ylabel('PC2')\n",
    "ax[1].set_ylim([-1, 1])\n",
    "ax[1].set_yticks([])\n",
    "ax[1].set_xlabel('PC1')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wow，结果非常好。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## 映射新的数据点\n",
    "\n",
    "在前面的两个例子中，我们将原始的数据集映射到新的特征空间。不过在实际应用中，我们常常需要将多个数据集转换，比如训练集和测试集，还有可能在训练好模型后，又收集到的新数据。在本节，你将学习如何将不属于训练集的数据进行映射。\n",
    "\n",
    "\n",
    "还记得在标准PCA中，我们通过计算 转换矩阵*输入样本，得到映射后的数据。转换矩阵的每一列是我们从协方差矩阵中得到的k个特征向量。现在，如何将这种思路应用到核PCA？在核PCA中，我们得到的特征向量来自归一化的核矩阵(centered kernel matrix)，而不是协方差矩阵，这意味着样本已经被映射到主成分轴$v$.因此，如果我们要把一个新样本$\\bf x^{'}$ 映射到主成分轴，我们要按照下式:\n",
    "\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/27/57711fdcd091a.png)\n",
    "\n",
    "上式怎么算？当然不好算，好在我们还有核技巧，所以可以避免直接计算$\\phi(x^{'})^{T}v$。\n",
    "\n",
    "和标准PCA不同的是，核PCA是一种基于内存的方法，这是什么意思呢？意思是每次对新样本进行映射时就要用到所有的训练集。因为要计算训练集中每个样本和新样本$x^{'}$之间的RBF核(相似度):\n",
    "\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/27/577120ee9abbe.png)\n",
    "\n",
    "其中，核矩阵$\\bf K$的特征向量$\\bf a$和特征值$\\lambda$满足条件: $\\bf K\\bf a=\\lambda \\bf a$。\n",
    "\n",
    "计算每一个训练集样本和新样本的$k()$后，我们必须用特征值对特征向量做归一化。所以呢，我们要修改一下前面实现的RBF PCA，能够返回核矩阵的特征向量："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy.spatial.distance import pdist, squareform\n",
    "from scipy import exp\n",
    "from scipy.linalg import eigh\n",
    "import numpy as np\n",
    "\n",
    "def rbf_kernel_pca(X, gamma, n_components):\n",
    "    \"\"\"\n",
    "    RBF kernel PCA implementation.\n",
    "    \"\"\"\n",
    "    # Calculate pairwise squared Eculidean distances\n",
    "    sq_dists = pdist(X, 'sqeuclidean')\n",
    "    mat_sq_dists = squareform(sq_dists)\n",
    "    #Compute the symmetric kernel matrix\n",
    "    K = exp(-gamma * mat_sq_dists)\n",
    "    #Center the kernle matrix\n",
    "    N = K.shape[0]\n",
    "    one_n = np.ones((N, N)) / N\n",
    "    K = K - one_n.dot(K) - K.dot(one_n) + one_n.dot(K).dot(one_n)\n",
    "    #Obtaining eigenpairs from the centered kernel matrix\n",
    "    eigvals, eigvecs = eigh(K)\n",
    "    alphas = np.column_stack((eigvecs[:, -i]\n",
    "    for i in range(1,n_components+1)))\n",
    "    #Collect the corresponding eigenvalues\n",
    "    lambdas = [eigvals[-i] for i in range(1, n_components+1)]\n",
    "    return alphas, lambdas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在，我们创建一个新的半月形数据集，然后用更新过的核PCA将其映射到一维子空间："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [],
   "source": [
    "X, y = make_moons(n_samples=100, random_state=123)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [],
   "source": [
    "alphas, lambdas = rbf_kernel_pca(X, gamma=15, n_components=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了检验对于新数据点的映射表现，我们假设第26个点时新数据点$x^{'}$，我们的目标就是将这个新数据点进行映射:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_new = X[25]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 1.8713,  0.0093])"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_new"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_proj = alphas[25]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.0788])"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_proj"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [],
   "source": [
    "def project_x(x_new, X, gamma, alphas, lambdas):\n",
    "    pair_dist = np.array([np.sum((x_new-row)**2) for row in X])\n",
    "    k = np.exp(-gamma * pair_dist)\n",
    "    \n",
    "    return k.dot(alphas/lambdas)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用project_x函数，我们能够对新数据样本进行映射："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.0788])"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_reproj = project_x(x_new, X, gamma=15, alphas=alphas, lambdas=lambdas)\n",
    "x_reproj"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最后，我们将第一主成分进行可视化："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAD8CAYAAAC2PJlnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3Xl4VOXd//H3l7AEECggArIIIi5AQoSI2LpRFG2tUkFb/aHF1j70sbY/Slv3y4Ji+1if/kRsXS66IGpVqnWhaqWoIMtTltDiho+CgKxKSCCQsAXy/f1xzkwmyWQ7M4EAn9d1zTVn7u3c98xkvueec+aOuTsiIiJRNDncHRARkSOXgoiIiESmICIiIpEpiIiISGQKIiIiEpmCiIiIRKYgIiIikaUliJjZpWb2sZmtNrPbk+S3MLOZYf4SM+sVpnc0s7lmVmxmv6tUZ17Y5orwdkI6+ioiIunTNNUGzCwDeAS4GNgILDOzWe6+MqHYjcB2dz/FzK4Bfg18G9gL3A0MCG+VjXH3vFT7KCIiDSPlIAIMAVa7+xoAM3sOGAkkBpGRwKRw+wXgd2Zm7l4CLDSzU9LQD44//njv1atXOpoSETlmLF++fJu7d4pSNx1BpBuwIeHxRuDs6sq4+wEzKwI6AttqaXu6mR0E/grc57Ws0dKrVy/y8jRxERGpDzP7LGrdxnxifYy7ZwHnhbfrkxUys3Fmlmdmefn5+Ye0gyIix7p0BJFNQI+Ex93DtKRlzKwp0A4oqKlRd98U3u8CniH42ixZuWnunuvuuZ06RZqNiYhIROkIIsuAvmbW28yaA9cAsyqVmQWMDbevAt6u6aspM2tqZseH282AbwAfpKGvIiKSRimfEwnPcfwImA1kAH9y9w/N7F4gz91nAX8EnjKz1UAhQaABwMzWAW2B5mb2TWAE8BkwOwwgGcCbwO9T7ascO0pLS9m4cSN79+493F0RaTQyMzPp3r07zZo1S1ubdjT9P5Hc3FzXiXUBWLt2LW3atKFjx46Y2eHujshh5+4UFBSwa9cuevfuXSHPzJa7e26UdhvziXWRyPbu3asAIpLAzOjYsWPaZ+cKInLUUgARqagh/iYUREREJDIFEZHQli1b6NOnD59//vkh3e/Xv/51duzYUWOZX/ziF7z55puR2p83bx7f+MY3ItWtTV36nsy8efP4n//5n/jjxx9/nCeffDKdXavilltuoX///txyyy0ptbN582auuuqqWsv96le/Spq+a9cu+vTpw6pVq4DgIpCsrCyWLFnCunXraNmyJTk5OQBs2LCBYcOG0a9fP/r378/UqVPj7UyaNIlu3bqRk5NDTk4Or7/+OgALFiygX79+DBiQbCWpBuDuR81t8ODBLuLuvnLlynrXuemmm7xJkyb+wx/+sAF6VFVZWZkfPHiwwfczd+5cv+yyyyLVbag+Tpw40f/7v/877e3WpG3btn7gwIFDtr/WrVtXmzdz5kwfMWKEu7v/6le/8nHjxrm7+9q1a71///7xcps3b/bly5e7u/vOnTu9b9++/uGHH7p7zc9h5XYSJfvbILiSNtLnrmYiIgSzkOnTp1NWVsb06dPTMht58MEHGTBgAAMGDOChhx4CYN26dZx22ml85zvfYcCAAWzYsIFevXqxbVuwAtDkyZM57bTTOPfcc7n22mv5zW9+A8ANN9zACy+8AATL+0ycOJFBgwaRlZXF//7v/wKwdOlSzjnnHM4880y+/OUv8/HHH9fYvyeeeIKRI0dy4YUX0rdvX+65555q+/jss8+SlZXFgAEDuO222+JtJPb96aefZsiQIeTk5PCDH/yAgwcPAvDGG28waNAgBg4cyPDhw1m3bh2PP/44U6ZMIScnhwULFjBp0qT4WFesWMHQoUPJzs7myiuvZPv27QBceOGF3HbbbQwZMoRTTz2VBQsWVBmTu3PLLbcwYMAAsrKymDlzJgBXXHEFxcXFDB48OJ4WM2nSJK6//nrOOecc+vbty+9///sa21q3bl38KP+JJ55g1KhRXHrppfTt25dbb70VgNtvv509e/aQk5PDmDFjqvTzW9/6FgAPPPAAjz/+OP/1X/+V9DXq2rUrgwYNAqBNmzacccYZbNpU+bfch1nU6NMYb5qJSEx9ZyI33XSTN2/e3AFv3rx5yrORvLw8HzBggBcXF/uuXbu8X79+/q9//cvXrl3rZub//Oc/42VPOukkz8/P96VLl/rAgQN9z549vnPnTj/llFPiR5pjx471559/Pl7+4Ycfdnf3Rx55xG+88UZ3dy8qKvLS0lJ3d58zZ46PGjXK3aufiUyfPt27dOni27Zt8927d3v//v192bJlVfq4adMm79Gjh2/dutVLS0t92LBh/tJLL1Xo+8qVK/0b3/iG79+/P/58zpgxw7du3erdu3f3NWvWuLt7QUGBu1c9ik58nJWV5fPmzXN397vvvtvHjx/v7u4XXHCB//SnP3V399dee82HDx9eZUwvvPCCX3TRRX7gwAH//PPPvUePHr5582Z3r35mMHHiRM/Ozvbdu3d7fn6+d+/e3Tdt2lRtW4lH+dOnT/fevXv7jh07fM+ePd6zZ09fv359jfuL+eijjxzwadOmxdNqmkGsXbvWe/To4UVFRfF+n3TSSZ6VleXf/e53vbCwsE7taCYikmaxWcj+/fsB2L9/f8qzkYULF3LllVfSunVrjjvuOEaNGhU/cj7ppJMYOnRolTqLFi1i5MiRZGZm0qZNGy6//PJq2x81ahQAgwcPZt26dQAUFRVx9dVXM2DAACZMmMCHH35Yaz8vvvhiOnbsSMuWLRk1ahQLFy6s0sdly5Zx4YUX0qlTJ5o2bcqYMWOYP39+hXbeeustli9fzllnnUVOTg5vvfUWa9asYfHixZx//vnx3yV06NChxv4UFRWxY8cOLrjgAgDGjh1bYV/Jxp1o4cKFXHvttWRkZNC5c2cuuOACli1bVuvzMHLkSFq2bMnxxx/PsGHDWLp0aZ3bGj58OO3atSMzM5N+/frx2Wd1W8vwjTfeoGvXrnzwQe2LcRQXFzN69Ggeeugh2rZtC8BNN93Ep59+yooVK+jatSs/+9nP6rTfdFMQkWPe5MmTKSsrq5B28OBBJk+e3CD7a926dcpttGjRAoCMjAwOHDgAwN13382wYcP44IMP+Nvf/lan3wNUvuQz9ri+fXR3xo4dy4oVK1ixYgUff/wxkyZNqlcbdZFs3OlQ3fNQnz7Vp1+bN2/m4YcfZunSpbz++uu899571ZYtLS1l9OjRjBkzJh5EATp37kxGRgZNmjThP/7jP1i6dGmd+5xOCiJyTKs8C4lJdTZy3nnn8fLLL7N7925KSkp46aWXOO+882qs85WvfCX+4V9cXMyrr75ar30WFRXRrVs3IPiuvi7mzJlDYWEhe/bs4eWXX+YrX/lKlTJDhgzhnXfeYdu2bRw8eJBnn302PlOIGT58OC+88AJbt24FoLCwkM8++4yhQ4cyf/581q5dG0+H4Pv9Xbt2VdlXu3btaN++fXzW9tRTT1XZV03OO+88Zs6cycGDB8nPz2f+/PkMGZJ07dYKXnnlFfbu3UtBQQHz5s3jrLPOitxWTLNmzSgtLU2aN2HCBO688066d+/Ogw8+yM0330zwrVJF7s6NN97IGWecwU9/+tMKeVu2bIlvv/TSS4fuaqxKFETkmJZsFhKTymxk0KBB3HDDDQwZMoSzzz6b73//+5x55pk11jnrrLO44ooryM7O5mtf+xpZWVm0a9euzvu89dZbueOOOzjzzDPrfJQ+ZMgQRo8eTXZ2NqNHjyY3t+rKF127duX+++9n2LBhDBw4kMGDBzNy5Mh4vpnRr18/7rvvPkaMGEF2djYXX3wxW7ZsoVOnTkybNo1Ro0YxcOBAvv3tbwNw+eWX89JLL8VPrCeaMWMGt9xyC9nZ2axYsYJf/OIXdX4OrrzySrKzsxk4cCBf/epXeeCBB+jSpUut9bKzsxk2bBhDhw7l7rvv5sQTT4zcVsy4cePIzs6ucmJ9zpw5rF+/nhtvvBEInov27dsnvcR50aJFPPXUU7z99ttVLuW99dZbycrKIjs7m7lz5zJlypQ69y2top5MaYw3nViXmMSTh8X7ir2srKxKmc2bN3tmZqYD1d5atmzpW7ZsqVK3rKzMi/cVp73fu3btcnf3kpISHzx4cPzyzoYwffp0v/nmmyPXP3DggHfo0CF+Mv1IdTguN65OTSfE09WOTqyL1EPJ/hIueOICJsyeUOXrgppmITHJZiPuzoTZE7jgiQso2V+S1v6OGzeOnJwcBg0axOjRo+OXdzZG/fv35/vf/35aV4Q91mVkZFBUVBT/sWEUCxYs4PLLL+f4449PY89qEDX6NMabZiISEzvaKisr8/F/H+9Mwsf/fXyFGUm3bt1qnIXEbt26dYvXqak9kSNBumci6fgf6yKNlpkx5ZLgu+KpS4IlI6ZcMgUzY+PGjfVqy8MZyNQlUxl/9vh4OyLHMgUROerVFEjqSgFEJDkFETkmpBJIFEBEqqcgIseMKIFEAUSkZro6S44psUAy/uzxTF0yNelVWzHHegBJXFwxFXVZxr7y0vCJHnzwQb73ve/FH//5z3/msssuA4KFKXv37s3jjz8eL9uvXz+ys7MZPnx4hSVIMjIy4r+1uOKKK+LpY8aMoUOHDvEFLqWeop6Rb4w3XZ0lMbUtwFjbVVbpvArrUC35nm6xxRUPhZp+q1FaWuoDBw70hQsX+vbt271Xr17+6aefunvFhSnd3d9++20vKSlxd/dHH33Uv/Wtb8XzaloQsXI7RzP9TkQkDWqakXgaZiDJllP/xz/+wTnnnMOgQYO4+uqrKS4uBoIj/jvuuIOcnBxyc3P517/+xSWXXEKfPn3iR9jFxcUMHz48vvz7K6+8Et/P6aefzpgxYzjjjDO46qqr2L17d7zd2K+ahwwZwurVqwHIz89n9OjRnHXWWZx11lksWrQIgIKCAkaMGBH//Ufs+ajsuOOOY8KECfTv35/hw4eTn58PVL+Ee23L2CdbGj5R06ZNefTRR7n55pu59dZb+d73vsfJJ5+ctG/Dhg2jVatWAAwdOrTeV+BJBFGjT2O8aSYiMXVdCj7ZjCMdM5DKy6nn5+f7eeed58XFwa/c77//fr/nnnvcPTjif/TRR93d/Sc/+YlnZWX5zp07fevWrX7CCSe4e3A0HlsCPD8/3/v06eNlZWW+du1aB3zhwoXu7v7d7343fkR/0kkn+X333efu7jNmzIgvB3/ttdf6ggUL3N39s88+89NPP93d3X/84x/H+/Tqq686kHQmAvjTTz/t7u733HNP/Ffv1S3hXpdl7Ovyq/FrrrnGe/fu7Xv37o2n1TSDuPnmm33y5MnxxxkZGT548GA/++yz40vZ16Wdo41+JyKSRpVPtsdOuKfjHEjicuqLFy9m5cqV8QUO9+/fzznnnBMvG/uOPisri+LiYtq0aUObNm1o0aIFO3bsoHXr1tx5553Mnz+fJk2asGnTJr744gsAevToEW/3uuuu4+GHH+bnP/85ANdee238fsKECQC8+eabrFy5Mr7vnTt3UlxczPz583nxxRcBuOyyy2jfvn3ScTVp0iS+BtZ1113HqFGjki7hfvXVVyetn7ice2x/tSkuLiYvL4/S0lLy8/Pp3r17jeWffvpp8vLyeOedd+Jpn332Gd26dWPNmjV89atfJSsriz59+tRp/1I9BRE55sUCSSyAQP1/R5JM4nLq7s7FF1/Ms88+m7RsbDnxJk2aVFhavEmTJhw4cIA///nP5Ofns3z5cpo1a0avXr3iS73XtIx5su2ysjIWL15MZmZmSuNLto+6iLKc+8SJE7nuuuvo3LkzEyZM4Pnnn6+27Jtvvskvf/lL3nnnnQrPZWyF45NPPpkLL7yQf//73woiaaBzInLM8/AcSKKartqKYujQoSxatCh+XqKkpIRPPvmkzvWLioo44YQTaNasGXPnzq1w1dH69ev55z//CcAzzzzDueeeG8+L/UvXmTNnxmc+I0aM4Le//W28zIoVKwA4//zzeeaZZwD4+9//Hj+nUVlZWVn8HEdsf6ku4V7d0vAA77//Pq+99hq33XYb48aNY926dcyZMydp2X//+9/84Ac/YNasWZxwwgnx9O3bt7Nv3z4Atm3bxqJFi+jXr1+d+yfVUxCRY1osgMROopf9oqxOl//WV6dOnXjiiSe49tpryc7O5pxzzon/b/S6GDNmDHl5eWRlZfHkk09y+umnx/NOO+00HnnkEc444wy2b9/OTTfdFM/bvn072dnZTJ06Nb5U+MMPP0xeXh7Z2dn069cvfvJ+4sSJzJ8/n/79+/Piiy/Ss2fPpH1p3bo1S5cuZcCAAbz99tvxpdpTWcK9uqXh3Z2bbrqJKVOmkJmZSZMmTXjssccYP358lf8BA3DLLbdQXFzM1VdfXeFS3o8++ojc3FwGDhzIsGHDuP322xVE0iXqyZTGeNOJdYmpy4n16k6iH0mLLNa05HdDXaJb2/8OP5TSdUJcJ9YP8yW+ZnapmX1sZqvN7PYk+S3MbGaYv8TMeoXpHc1srpkVm9nvKtUZbGbvh3UetmPpV17S4Nyrv4y3Pj9IlMOrXbt23H333fHZVBRjxozhnXfeSds5omNO1OgTuwEZwKfAyUBz4F2gX6UyPwQeD7evAWaG262Bc4H/BH5Xqc5SYChgwN+Br9XWF81EJKammUhdZxpH0oxEpK4a4yW+Q4DV7r4GwMyeA0YCKxPKjAQmhdsvAL8zM3P3EmChmZ2S2KCZdQXauvvi8PGTwDfDYCJSJ+5e5cohr2EGUlk6Vv8VaUy8AWbU6Qgi3YANCY83AmdXV8bdD5hZEdARqG5hnm5hO4ltdktW0MzGAeOAak8EyrEnMzOTgoICOnbsGP/Qr08AiVEgkaOFu1NQUJD2r+2O+N+JuPs0YBpAbm6uvrgWALp3787GjRvjS3K4O/evuJ+nVj3F9X2vZ1zPcfW6Ompcz3EUFhYydclUCgsLuT3ndgUSOeJkZmbW+kPN+kpHENkE9Eh43D1MS1Zmo5k1BdoBBbW0mTjSZG2KVKtZs2b07t0bKJ+BPLXqqZR+iT7jjBl0mN2BqUum0qFDB81IREhPEFkG9DWz3gQf9NcA/6dSmVnAWOCfwFXA217Dl3PuvsXMdprZUGAJ8B3gt9WVF6lOlK+wqqOvtkSqSjmIhOc4fgTMJrhS60/u/qGZ3Utwxn8W8EfgKTNbDRQSBBoAzGwd0BZobmbfBEa4+0qCK7qeAFoSnFDXSXWpt92lu1m4fmHa/h9IYiBZuH4hu0t307p561pqiRy9rCHO1h8uubm5npeXd7i7IY1Myf4SWjVrldYZg7srgMhRw8yWu3tulLpH/Il1kdo0xAe9mSmAiKC1s0REJAUKIiIiEpmCiIiIRKYgIiIikSmIiIhIZAoiIiISmYKIiIhEpiAiIiKRKYiIiEhkCiIiIhKZgoiIiESmICIiIpEpiIiISGQKIiIiEpmCiIiIRKYgIiIikSmIiIhIZAoiIiISmYKIiIhEpiAiIiKRKYiIiEhkCiIiIhKZgoiIiESmICIiIpEpiIiISGQKIiIiEllagoiZXWpmH5vZajO7PUl+CzObGeYvMbNeCXl3hOkfm9klCenrzOx9M1thZnnp6KeIiKRX01QbMLMM4BHgYmAjsMzMZrn7yoRiNwLb3f0UM7sG+DXwbTPrB1wD9AdOBN40s1Pd/WBYb5i7b0u1jyIi0jDSMRMZAqx29zXuvh94DhhZqcxIYEa4/QIw3MwsTH/O3fe5+1pgddieiIgcAdIRRLoBGxIebwzTkpZx9wNAEdCxlroO/MPMlpvZuOp2bmbjzCzPzPLy8/NTGoiIiNRPYz6xfq67DwK+BtxsZucnK+Tu09w9191zO3XqdGh7KCJyjEtHENkE9Eh43D1MS1rGzJoC7YCCmuq6e+x+K/AS+ppLRKTRSUcQWQb0NbPeZtac4ET5rEplZgFjw+2rgLfd3cP0a8Krt3oDfYGlZtbazNoAmFlrYATwQRr6KiIiaZTy1VnufsDMfgTMBjKAP7n7h2Z2L5Dn7rOAPwJPmdlqoJAg0BCW+wuwEjgA3OzuB82sM/BScO6dpsAz7v5Gqn0VEZH0smBCcHTIzc31vDz9pEREpD7MbLm750ap25hPrIuISCOnICIiIpEpiIiISGQKIiIiEpmCiIiIRKYgIiIikSmIiIhIZAoiIiISmYKIiIhEpiAiIiKRKYiIiEhkCiIiIhKZgoiIiESmICIiIpEpiIiISGQKIiIiEpmCiIiIRKYgIiIikSmIiEijVbK/hHT/C293p2R/SVrbPJYpiIhIo1Syv4QLnriACbMnpC2QuDsTZk/ggicuUCBJEwUREWmUWjVrxbk9z2XqkqlpCSSxADJ1yVTO7XkurZq1SlNPj21ND3cHRESSMTOmXDIFgKlLpgIw5ZIpmFm920oMIOPPHh+5HalKQUREGq10BBIFkIalICIijVoqgUQBpOEpiIhIoxclkCiAHBoKIiJyRKgpkGzZsoVzzz2XRYsW0aVLFwWQQygtV2eZ2aVm9rGZrTaz25PktzCzmWH+EjPrlZB3R5j+sZldUtc2ReTYEwsk488eX+GqrcmTJ7Nu3TomT56sAHKIpTwTMbMM4BHgYmAjsMzMZrn7yoRiNwLb3f0UM7sG+DXwbTPrB1wD9AdOBN40s1PDOrW1eeh98gl87WvQuTNkZsKDD8Kdd8LDD8Ozz8L+/eAe5P3kJ8H2Qw+BGdx4Izz1FFx/ffL7H/4wKP/YY9HSjjsOdu2qOT2xbm11oG55NbWZLK+2uvXNr7y/dO0jVib2+v3kJxXzEiXbfxq99x68+CKsXw89e8KoUZCdXbXMY4/B4sXBcIYODbqTnV2xfvPmwXD27StvC8rzW7QI6u/fX3FfiW0kK5PYRpR91FR/wAD44IPEusa+/VM4u2cwI9mwoYRXfv8UZWVlPPb4n3j14H7Wd/0DY/qM57tdp3DPPcaKFbBjB3zpS5CTU7HN6vZXXZ3E9C5dgrpbtpSnde0ajO/zz6svZxaUSbzftQuKi6FlS+jbN0h/910oLAyen+bNy5+3PXuC/paVBXkdO8Idd8DPfpb2t1+t0vF11hBgtbuvATCz54CRQOIH/khgUrj9AvA7Cw4NRgLPufs+YK2ZrQ7bow5tHnp33QXr1gW3zEz40Y9g1aogfc2a4N0Bwbtm4MDgFZ81K0jbuzd4R+zcmfz+jDOC8gsWREu7/HKYO7fm9MS6tdWBuuXV1GayvNrq1je/8v7StY9YmdjrN3BgxbxEyfafJu+9B7/5DbRvD927w/btweOf/7w8kLz3XvAWXL0a2rQJ0t55BzZuhBtuCIbQvj00axakA5x/ftDWXXcFw+/TJ8ifN69i/m9+A1dcUbGNymUqt1Hffdx5Z/AhevLJVet/8gk8+SScc04Qn8vrGkO2T2FrKbzIVLioCbwBPiIIIKdtH8+OmVO46zmjXbvgz7NJk+ADed++im0m7m/VqiDvtNNg06byOqWlQfrppwfPa5MmQf677wZ1mzaF1q1h8+aqaYnlmjULxvrFF8HHRFERtG0bPA/uQZBs1w5eey0IEE2bBkHj4MHgFgs6lRUUBM8jHPpAko6vs7oBGxIebwzTkpZx9wNAEdCxhrp1afPQ+uQTmD07eAXLyoJXdskSOP54eOON4IN227bgtns3/OUv8PzzwTt29+5g+8QT4a9/Lb/v1i247907OAx7+WXo1StI69Wralp15V59NTjEee216tNPPbW87qmnJq/Tu3eQXlwcHBbVlldTm8nyaqtb3/zK+0vXPiAo89JLweu3b1+wHctLVLk/ycqk4MUXgw/v9u2DD67Y9osvViyzdWvwYdSqVXBr2xby8+F3vyuv8/HHQXrbtsF2+/ZBvfz86vPbt0+9jdry8/ODMsnyN28OtjdtqprXob3R9p+3wuIMGFoWHKYOLYPFGbRfehv5W438/KBuy5bBh3bLlvDRR9W3uWlTsP3RRxXrfPhhkL5yZXn67t3Bx8D+/cH2l74EJSVV0xLLlZQEb6fMzOCYs3nzIJAcPBi8li1aBAHBDA4cCOrEZiDVBZCYgwfht79N69uvTo74E+tmNg4YB9CzZ8+G29FddwWziZgDB4L7rVuDd8UXXwRpZsG7JnbokZERfNDs2xe8M0tLy+9XrgzuY39FACecEKRt2xakmUGnTjWXy8yERx4J3nE7dyZPb9MmmIebBXP2wsKqdfLzgzpz55bPm2vKq6nNWF7wwgR5tdWtb37l/UXdR+U+xmY369cHr597sB3LSzR3btX9p3E2sn59MANJ1K5dkJ5YZt++ID0mMzP4cCosDI6wofyoN7YNQb2YZPnt2gUfrKm0kWp+27YVtxPzNm64D1Y0gaEHyxuZ3YRVHe6je/dHgODlidXLzAz+HHv2rL7Ntm1hwwY46aTyOjt3BnUS0w8eLP9wj4l9BCSmJ5ZzDx63aFE+1p07gwME92DmsW9f+dvuwIFg/3X9sX7sq69DKR1BZBPQI+Fx9zAtWZmNZtYUaAcU1FK3tjYBcPdpwDSA3Nzc9K7UFhObhRw4UPXVLCgIDhV27gzeGU2aBO/S2Befp5wSfIF54EBwONOrVxA8evYM7k8+OQgqZWVB2wUFwezmo4/KD08S0yqX++STYF7+17/C8OHBdxux9C9/OUj/+teDd2ZBQdDe/v3BYV+yOhdeGBzamgVl3nkneV6XLtW3mWx/XbrUXLe++VBxf5Cefbz6KuTmBjOPgoLycxzbtgVpw4ZVPK/y2mvl/YnVTyyTop49g6862rcvTysqCtITy3z4YXCM07JlkLZ3b/B2jH1l0r59EBD27AnyYwGnRYvydpLlFxUFE+ZU2kg1P3ZeASrm7dq1hYLCP8GlpRWftEtKKfzHn+jR426aN+9Cy5ZBvZYtg+elTZvgzzVZm7H9tWlT/nzu3Vv+YZ+YnpFBFU2TfKJWLtesWfnrs29fcF9aWj77aNGiPBhlZAR/8rXNQmI6dKi9TLql4+usZUBfM+ttZs0JTpTPqlRmFjA23L4KeNuDhXBmAdeEV2/1BvoCS+vY5qEB+vGQAAAOI0lEQVRTeRaSyD14B0D5K79nT3ArLQ3OrsXy9u0LPozKysrvd+8O/kKLi4O5blFR+aFPSUmQHktLVq6sLAgCpaXB/cGD5envvhukb94cnMeJHRqtXRt8DRSrU1YW1Dl4MCi7fj189llQprq8Fi2qbzO2Pwjy164NytdUt775UHF/EG0flfu4f38wu1m/Pkhv2rT8kyE2G4mJzUJi/YnVTyyTolGjgiCyfXvwUsS2YyejY2VOOCF4y+zeHdx27gwmsD/6UXmd004L0nfuDLa3bw/qdepUff727am3UVt+p05BmWT5J54YbHfrVjXvH3PuhUtKYSiwmODrrMXA0ODcSOH2e+nUKai7Z08QHPbsCU5dVddmt27B9hlnVKzTv3+Q3q9feXqrVsHxY/PmwfaOHcE5kMppieVatw7eJnv3BkFs//4geMUCzb59wUny2KykefOgTOXZTTIZGfDjH6ftrVdnlo7VMc3s68BDQAbwJ3f/pZndC+S5+ywzywSeAs4ECoFrEk6a3wV8DzgA/MTd/15dm7X1Izc31/Py8lIeTxUnnxx88MQuhahJbB4a287IKJ+jxtKaNg2CStOmwWFJZR06VJ2XVpcGQVBp3br8Pib2OHa2ddeu4D7xceU6sbyYWJ3KeQMHBkEqWZvJ9hcrX13d+uZDeZlYfiwtlX1A8J1FYWHFscfKjBoFd98dPJ48GT79lCr69Ckvkwa6Oqvy1Vmwfcdmnt/ZEz/7YBA43kh4Mi4FhkLGsgxmT9jAggVddXVWLcxsubvnRqqb7rX6D6cGCyIi0mi4OwNvG8j7rd+vGkBiwkCSvTubFfev0O9EapFKENFS8CJyxIj9kLDGAEKYvhjea/VeWv8fiVSlICIiR4TKv0Qve70Md6/2VvZ6WZVftkv6HfGX+IrI0S/KUibp/H8kUj0FERFp1FJZC0uBpOEpiIhIo5WOxRQVSBqWgoiINErpXI1XgaThKIiISKO0u3Q3C9cvTNty7omBZOH6hewu3U3r5q1rqSW10e9ERKTRKtlfQqtmrdI6Y3B3BZBKUvmdiGYiItJoNcQHvZkpgKSRficiIiKRKYiIiEhkCiIiIhKZgoiIiESmICIiIpEpiIiISGQKIiIiEpmCiIiIRKYgIiIikSmIiIhIZAoiIiISmYKIiIhEpiAiIiKRKYiIiEhkCiIiIhKZgoiIiESmICIiIpEpiIiISGQpBREz62Bmc8xsVXjfvppyY8Myq8xsbEL6YDN738xWm9nDFv4jZTObZGabzGxFePt6Kv0UEZGGkepM5HbgLXfvC7wVPq7AzDoAE4GzgSHAxIRg8xjwH0Df8HZpQtUp7p4T3l5PsZ8iItIAUg0iI4EZ4fYM4JtJylwCzHH3QnffDswBLjWzrkBbd1/s7g48WU19ERFppFINIp3dfUu4/TnQOUmZbsCGhMcbw7Ru4Xbl9Jgfmdl7Zvan6r4mExGRw6vWIGJmb5rZB0luIxPLhbMJT1O/HgP6ADnAFuD/1dC/cWaWZ2Z5+fn5adq9iIjURdPaCrj7RdXlmdkXZtbV3beEX09tTVJsE3BhwuPuwLwwvXul9E3hPr9I2MfvgVdr6N80YBpAbm5uuoKYiIjUQapfZ80CYldbjQVeSVJmNjDCzNqHX0uNAGaHX4PtNLOh4VVZ34nVDwNSzJXAByn2U0REGkCtM5Fa3A/8xcxuBD4DvgVgZrnAf7r799290MwmA8vCOve6e2G4/UPgCaAl8PfwBvCAmeUQfD22DvhBiv0UEZEGYMGpjKNDbm6u5+XlHe5uiIgcUcxsubvnRqmrX6yLiEhkCiIiIhKZgoiIiESmICIiIpEpiIiISGQKIiIiEpmCiIiIRKYgIiIikSmIiIhIZAoiIiISmYKIiIhEpiAiIiKRKYiIiEhkCiIiIhKZgoiIiESmICIiIpEpiIiISGQKIiIiEpmCiIiIRKYgIiIikSmIiIhIZAoiIiISmYKIiIhEpiAiIiKRKYiIiEhkCiIiIhKZgoiIiESWUhAxsw5mNsfMVoX37aspNzYss8rMxiak/9LMNphZcaXyLcxsppmtNrMlZtYrlX6KiEjDSHUmcjvwlrv3Bd4KH1dgZh2AicDZwBBgYkKw+VuYVtmNwHZ3PwWYAvw6xX6KiEgDSDWIjARmhNszgG8mKXMJMMfdC919OzAHuBTA3Re7+5Za2n0BGG5mlmJfRUQkzVINIp0TgsDnQOckZboBGxIebwzTahKv4+4HgCKgY2pdFRGRdGtaWwEzexPokiTrrsQH7u5m5unqWF2Z2ThgHEDPnj0P9e5FRI5ptQYRd7+oujwz+8LMurr7FjPrCmxNUmwTcGHC4+7AvFp2uwnoAWw0s6ZAO6Cgmv5NA6YB5ObmHvIgJiJyLEv166xZQOxqq7HAK0nKzAZGmFn78IT6iDCtru1eBbzt7goQIiKNTKpB5H7gYjNbBVwUPsbMcs3sDwDuXghMBpaFt3vDNMzsATPbCLQys41mNils949ARzNbDfyUJFd9iYjI4WdH0wF+bm6u5+XlHe5uiIgcUcxsubvnRqmrX6yLiEhkCiIiIhKZgoiIiESmICIiIpEpiIiISGQKIiIiEpmCiIiIRKYgIiIikSmIiIhIZAoiIiISmYKIiIhEpiAiIiKRKYiIiEhkCiIiIhKZgoiIiESmICIiIpEpiIiISGQKIiIiEpmCiIiIRKYgIiIikSmIiIhIZAoiIiISmYKIiIhEpiAiIiKRKYiIiEhkCiIiIhKZgoiIiESWUhAxsw5mNsfMVoX37aspNzYss8rMxiak/9LMNphZcaXyN5hZvpmtCG/fT6WfIiLSMFKdidwOvOXufYG3wscVmFkHYCJwNjAEmJgQbP4WpiUz091zwtsfUuyniIg0gFSDyEhgRrg9A/hmkjKXAHPcvdDdtwNzgEsB3H2xu29JsQ8iInKYpBpEOicEgc+BzknKdAM2JDzeGKbVZrSZvWdmL5hZjxT7KSIiDaBpbQXM7E2gS5KsuxIfuLubmaepX38DnnX3fWb2A4JZzler6d84YBxAz54907R7ERGpi1qDiLtfVF2emX1hZl3dfYuZdQW2Jim2Cbgw4XF3YF4t+yxIePgH4IEayk4DpoX9yTezz2pqO8HxwLY6lj2SHI3jOhrHBEfnuDSmI0fiuE6K2kitQaQWs4CxwP3h/StJyswGfpVwMn0EcEdNjcYCU/jwCuCjunTG3TvVpVy4jzx3z61r+SPF0Tiuo3FMcHSOS2M6cqRrXKmeE7kfuNjMVgEXhY8xs1wz+wOAuxcCk4Fl4e3eMA0ze8DMNgKtzGyjmU0K2/2/Zvahmb0L/F/ghhT7KSIiDcDc03Ua48iio4sjx9E4Jjg6x6UxHTkay0zkSDbtcHeggRyN4zoaxwRH57g0piNHWsZ1zM5EREQkdcfyTERERFJ0VAeReqzt9YaZ7TCzVyulP2FmaxPW8Mo5ND2vWRrG1dvMlpjZajObaWbND03Pq5eGddjmmdnHCa/VCYeu91X6eGnYl9VmlmwpoBbh8746fB16JeTdEaZ/bGaXHMp+1yTqmMysl5ntSXhdHj/Ufa9JHcZ1vpn9y8wOmNlVlfKSvhcPtxTHdDDhtZpVpx26+1F7I/h9ye3h9u3Ar6spNxy4HHi1UvoTwFWHexwNMK6/ANeE248DNx0JYwI6AGvC+/bhdvswbx6Q2wjGkQF8CpwMNAfeBfpVKvND4PFw+xqCdeIA+oXlWwC9w3YyjvAx9QI+ONxjSGFcvYBs4MnEz4Ka3otH6pjCvOL67vOonolQt7W9cPe3gF2HqlNpEHlcZmYEv/5/obb6h1hK67A1IkOA1e6+xt33A88RjC1R4lhfAIaHr8tI4Dl33+fua4HVVL9A6aGUypgas1rH5e7r3P09oKxS3cb6XkxlTJEc7UGkLmt71eaXFqzhNcXMWqSxb6lIZVwdgR3ufiB8XNe1zBpaOtZhmx5Ow+8+jB9gdVkrLl4mfB2KCF6XqOvMNbRUxgTQ28z+bWbvmNl5Dd3Zekjl+T6SX6uaZJpZnpktNrM6HVym+ov1w84adm2vOwg+0JoTXA53G3BvlH7WVwOP67Bo4DGNcfdNZtYG+CtwPcF0XQ6vLUBPdy8ws8HAy2bW3913Hu6OSVInhX9HJwNvm9n77v5pTRWO+CDiqa/tVVPbsSPjfWY2Hfh5Cl2tlwYcVwHwJTNrGh4xdidY36zBpWFM1a7D5u6bwvtdZvYMwbT+cASRTUDiqtPJnt9YmY1m1hRoR/C61KXu4RB5TB580b4PwN2Xm9mnwKlAXoP3unapPN/1XhPwEEnpPZTwd7TGzOYBZxKcY6nW0f51VmxtL6h+ba9qhR9msfMI3wQ+SGvvoos8rvCPei4Quyqj3s9LA6nLmGYDI8ysfXj11ghgtpk1NbPjAcysGfANDt9rtQzoG14B15zgJHPlq1wSx3oV8Hb4uswCrgmvdOoN9AWWHqJ+1yTymMysk5llAIRHt30JTkI3BnUZV3WSvhcbqJ/1EXlM4VhahNvHA18BVtZa8XBfTdCQN4LvZN8CVgFvAh3C9FzgDwnlFgD5wB6C7xAvCdPfBt4n+EB6GjjucI8pTeM6meDDaTXwPNDiCBrT98J+rwa+G6a1BpYD7wEfAlM5jFc1AV8HPiE4grsrTLsXuCLczgyf99Xh63ByQt27wnofA1873K9LqmMCRoevyQrgX8Dlh3ss9RzXWeHfTgnBbPHDmt6LjeEWdUzAl8PPu3fD+xvrsj/9Yl1ERCI72r/OEhGRBqQgIiIikSmIiIhIZAoiIiISmYKIiIhEpiAiIiKRKYiIiEhkCiIiIhLZ/wfcFMbMODHB+wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff3a41c1b50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(alphas[y==0, 0], np.zeros((50)),\n",
    "           color='red', marker='^', alpha=0.5)\n",
    "plt.scatter(alphas[y==1, 0], np.zeros((50)),\n",
    "           color='blue', marker='o', alpha=0.5)\n",
    "plt.scatter(x_proj, 0, color='black',\n",
    "           label='original projection of point X[25]',\n",
    "           marker='^', s=100)\n",
    "plt.scatter(x_reproj, 0, color='green',\n",
    "           label='remapped point X[25]',\n",
    "           marker='x', s=500)\n",
    "plt.legend(scatterpoints=1)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## sklearn中的核PCA\n",
    "\n",
    "sklearn.decomposition中有核PCA的实现，看看怎么用："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.decomposition import KernelPCA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [],
   "source": [
    "X, y = make_moons(n_samples=100, random_state=123)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [],
   "source": [
    "scikit_kpca = KernelPCA(n_components=2, kernel='rbf', gamma=15)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_skernpca = scikit_kpca.fit_transform(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过kernel参数设定不同的核函数。\n",
    "\n",
    "将转换后的数据可视化："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEKCAYAAAA8QgPpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3X/wXXV95/HnC2hSaxCCpIAQCGJwjTRN9SvaTrEy4BS2Q6KRKlo7scBmaGT6w2W37FocCt1ZRLDrrAyaWkd0x0FUWFKgsoI4y84Y5EsXvxJdSqAQEhG/EghRLIHy3j/OueTm5tzf957zOfe+HjPf+d577sm972++yXnfz+f9eX+uIgIzM7N+HVB1AGZmVk9OIGZmNhAnEDMzG4gTiJmZDcQJxMzMBuIEYmZmA3ECMTOzgVSaQCSdIelBSVslXdzhvPdICkkzZcZnZmbtVZZAJB0IXAOcCawA3i9pRcF5BwN/CtxTboRmZtbJQRW+9snA1oh4BEDS9cAa4Act510OfBz4D7086eGHHx7Lli0bYZhmZpPvvvvu+2lELOnnz1SZQI4GHm+6vx14a/MJkt4ELI2IWyX1lECWLVvG7Ozs6KI0M5sCkh7r988kW0SXdADwSeDf93Duekmzkmbn5+fHH5yZmVWaQHYAS5vuH5MfazgYOAn4tqRHgbcBm4oK6RGxMSJmImJmyZK+RmBmZjagKhPIvcByScdLWgCcA2xqPBgRuyLi8IhYFhHLgM3A6ojw/JSZWQIqSyAR8SJwIXA78EPghojYIukySaurisvMzHpTZRGdiLgNuK3l2MfanPuOMmIyM7PeJFtENzOztDmBmJnZQJxAzMxsIE4gZmY2kEqL6Dak3bvh2mthwwZYtKj3x8zsZXNzcOONsG0bHHssrF0LK1d2f8w8Aqm3u+6Cu+/Ovvf62O7dcOWV8LOflROjWSLm5uDSS+Hcc7Pvc3PZ11VXwdNPwzHHZN+vuqr7Y5bxCKQOikYTu3fDrbfCiSfCLbfAqaf29lgjsbzhDXDWWdX8PGYlaySDxYv3TQavfGV2bPHi7LzG9xtv3Hu/3WMemXgEUg9Fo4m77oI9e+Dgg7PvvTzWmlhaRyEenVjNFY0yILvYN5LBAQfsvb15MxxyyL7PccghWWLYtq34sfvv98ikwQkkdUUX/caxI4/MzjnyyN4e65R0oPOUmFniOk05tUsGEbBr177Hd+3KRhXHHlv82DPPFCejxshkmjiBpKZ1FFB00W8cW7gwO2fhwu6P3Xpr+8TSeN1OoxOzxLUbZdx4Y/tk8La3ZYnm6afhpZf23l67NvsqeuzQQ9uPTIpGP5PMCSQ1zaOAdqOJ7343e+v06KN7vyKyf8Hf+17xYzff3D7pNF630+jELHHtRhnbtrVPBhs2wEUXZYlm+/bs+0UXZfWMlSuLH1u1av9ktHUr/PM/T9+0lovoKWkdBTz3XPFF/61vhb/+6/6e+/LL4eGHs4TS7P774R3vKE5UzcX3RnxeGmwV6rSs9thjswt3o9gNe6ejGsmg+c+ed97eP9uuAN5IJK2uuir7fsgh2Wts2QInnVRccJ/k4roTSEqaRwE7d8Lf/z0cdFDxRb/fFVSXXNL+sU2b2o9Oml/HK7isQu1WUjVGDGvX7n9hf/rpLFFA+2TQr6JkdPzxcMIJ+57XGP1MMieQKjW/o4/YfxSwcydcffX43+03T3s1a05UnZYGm5WgucYB+7/L7zbKGKXWZHTppcWjn4ULs8cmdbmvE0iVmt/RR/Q2ChiHTqOT5libR0dlxGXWZNu2bOTRrPVd/qhGGf0qGv088kj233rBguIR0yRwEb0qre/oOxXGq9ZpaXDzOe4hsRFo18vRbiXVsceWHeH+igruRx+dTWtN8nJfj0Cq0vqOfpDCeFk6LRtujEJcH7ER6FTn6FbjqFrr6Ofcc+FXf3XfcyatLuIRSBV6eUefknZLgxujI/eQ2Ih06uVot6w21emglEdMo1LpCETSGcCngAOBz0XEFS2PXwB8GPhX4GfA+oj4QemBjkJzwbyXd/Qp6VYjcX3ERqRbnaOqGscg2o2YTjllcgrrlY1AJB0IXAOcCawA3i9pRctpX46IX4uIVcCVwCdLDnN0mhsEu72jr5O6jaYsaZP0rr1oxLR6dbZqflIaDqscgZwMbI2IRwAkXQ+sAV4eYUTEs03nvxKIUiMcldYpnjKW5palbqMpS1rqdY5+FS337bQUuW6qrIEcDTzedH97fmwfkj4s6WGyEcifFD2RpPWSZiXNzs/PjyXYoUzyNiG9jqa8Ssty7VZZQf3qHP3qtN1KHSW/CisirgGukfQB4C+BdQXnbAQ2AszMzKQ1Smk3xTMpjXi99JCAV2kZ0L2bHOpV5+hXp+1W6qjKEcgOYGnT/WPyY+1cD7xrrBGNQ6cpnmnhVVqW67TKahq029Rx7dqqIxtMlQnkXmC5pOMlLQDOATY1nyBpedPd3wMeKjG+0ZikgvmgJnkKz/oyaVM4/Zq0KbrKprAi4kVJFwK3ky3j/XxEbJF0GTAbEZuACyWdDrwAPE3B9FWyGst2//zPJ2OqalCTPoVnfZm0KZxBtE7RNWpCdVzWW2kjYUTcFhEnRsQJEfFf8mMfy5MHEfGnEfHGiFgVEadGxJYq4+2LP90v4yk8azJpUzjD6vQpinXgTvRx8Jz/Xp7CsyaTNoUzrLrXhJJfhVVL7szeq9dVWv6wqonR6UOfYLJXWfWrlx2GU+YRyKi5M3swnvKbCHWfkilb3TvvnUBGzXP+/fOU38So+5RM2epeE3ICGTXP+ffPy3wnxrQv0+1X3WtCroGMipftDsbLfCeKl+n2r87Lej0CGRXP4Q/GU34Tpe5TMlWrWw3JCWQUPIc/OE/5TZS6T8lUrW41JE9hjYKX7Q6u12W+Vhtepju4ui3r9QhkWF62Wx5vCV+5Tlux2/DqtqzXCWRYnsMvj+tMlarb/Hwd1a2G5AQyLM/hl8N1psrVbX6+jupWQ3INZFiewy+H60yVq9v8fF3VqYbkEcgwPCdfDteZklC3+flJkHrNyQlkGJ6TL4frTEmo2/x83dWh5uQEMijPyZfHdaYk1G1+vu7qUHNyDWRQnpMvj+tMyajT/Hzd1aHmVOkIRNIZkh6UtFXSxQWPf0TSDyTNSbpT0nFVxLkfz8mnyTWpoaU+5z5N6lBzqiyBSDoQuAY4E1gBvF/SipbT/i8wExErga8BV5YbZRuek0+Ta1JDqcOc+zSpQ82pyhHIycDWiHgkIvYA1wNrmk+IiLsi4rn87magZUBXEc/Jp8c1qaHVYc59mtSh5lRlDeRo4PGm+9uBt3Y4/zzgH8YaUa88J58e16SGVoc592mTes2pFquwJH0QmAE+0ebx9ZJmJc3Oz8+XG5xVzzWpkajDnLulpcoEsgNY2nT/mPzYPiSdDnwUWB0Rzxc9UURsjIiZiJhZsmTJWIJ9mQu16XFNaiTqMOc+zVJc4FBlArkXWC7peEkLgHOATc0nSPoN4LNkyeMnFcS4Pxdq0+Oa1EjUYc59WqW6wKGyGkhEvCjpQuB24EDg8xGxRdJlwGxEbCKbsloEfFUSwLaIWF1VzPsVav2xq2lwTWpkUp9zn1bNCxxg7/cbb6z291VpI2FE3Abc1nLsY023Ty89qE5cqDWzCqS6wKEWRfQkuFBrZhVJdYGDE0ivXKidDFO8CCLFIqz1JtUFDk4gvXKhdjJM6SKIVIuw1ptUFzh4M8VeuVBbf1O8CCLVIqz1LsUFDh6B2PRoXgQxZdOP27ZlRddmKRRhrd6cQGw6TPkiiFSLsFZvTiC9muLi60SY8kUQqRZhrX8pLYZwAunVlBZfJ8aUL4JItQhr/UltMYSL6L2Y4uLrxPAiiCSLsNaf1BZDeATSiykuvppZOlJbDOEE0s2UF1+nUg3rXSnNi9v4pLYYwgmkmykvvk6lmtW7UpsXt/FJbTGEE0g3U158nTo1/GhcfxTt9EhtMYSL6N24+Dpdarjjcqo7tdp4pLQYwiMQs4aa1rtSmxe36eEEYtZQ03pXavPiNj2cQMwaalrvSm1e3KaHIqLqGEZqZmYmZmdnR/eEu3fDtdfChg1uHjSzZMzNZQsltm3LpivXrh3uTYOk+yJipp8/U+kIRNIZkh6UtFXSxQWPv13SP0p6UdLZVcRYtyWdZjb5Ulm6XVkCkXQgcA1wJrACeL+kFS2nbQM+BHy53OhyNVzSaSUqoeHQDYJWJJWl21WOQE4GtkbEIxGxB7geWNN8QkQ8GhFzwEtVBOgtTKyjMY9OU3mXaelJZUuTKhPI0cDjTfe358fSUNMlnVaSEkanqbzLtPSksnR7IlZhSVovaVbS7Pz8/GietKZLOq0kJYxOU3mXaelJZel2lQlkB7C06f4x+bG+RcTGiJiJiJklS5aMJLi6Lum0EpQ0Ok3lXaalJ5Wl21VuZXIvsFzS8WSJ4xzgAxXGsy9vYWLtdBqdjnDbk7Vrs5oHZCOPXbuyd5nnnTeyl7AaS2FLk8pGIBHxInAhcDvwQ+CGiNgi6TJJqwEkvUXSduD3gc9K2lJVvGYvK2l0msq7TLN23EhoZmYDNRJ6N95O3IVuZgkbdTd6vyZiFdbYuAvdzBKVQp+QE0g77kI3s4Sl0CfkBNKOu9DNLGEp9Ak5gRRxF7qZJS6FPiEnkCLuQjezxKXQje4EUsRd6GaWuBT6hNwHYmZm9ftAKTMzqy8nEDMzG4gTiJmZDcRbmXTirUzMLGHeyiRl3srEzBLlrUxS5q1MzCxh3sokZd7KxMwSVoutTCS9StIJBccn92NtvJWJmSUu+a1MJL0X+H/A1yVtkfSWpoe/MM7AKuWtTMwscXXYyuQ/A2+OiFXAHwFfkvTu/DGNNbIqeSsTM0tcCluZdFvGe2BEPAEQEd+VdCpwi6SlwNB7oEg6A/gUcCDwuYi4ouXxhcAXgTcDTwHvi4hHh33dri65ZOwvYWY2rJUry00YrbqNQHY31z/yZPIOYA3wxmFeWNKBwDXAmcAK4P2SVrScdh7wdES8Dvgb4OPDvKaZmY1OtwTyx7RMVUXEbuAM4NwhX/tkYGtEPBIRe4DryRJTszXAdfntrwGnSZrcqTMzsxrplkB+DhxRcPxkYPOQr3008HjT/e35scJzIuJFYBfw6iFf18zMRqBbAvlvwLMFx5/NH0uCpPWSZiXNzs/PVx2OmdlU6JZAjoiI77cezI8tG/K1dwBLm+4fkx8rPEfSQcAhZMX01ng2RsRMRMwsWbJkyLByu3fDlVe698PMkjY3B5deCueem31PaSuTQzs89oohX/teYLmk4yUtAM4BNrWcswlYl98+G/hWlPUJWN4Hy8wSV/V+WN0SyKykf9d6UNL5wH3DvHBe07gQuB34IXBDRGyRdJmk1flpfwe8WtJW4CPAxcO8Zs+8D5aZ1UDV+2F16wP5M+AmSX/A3oQxAywA3t32T/UoIm4Dbms59rGm2/8C/P6wr9O35n2wdu7M7p91VulhmJl1sm1bNvJoVuZ+WB1HIBHxZET8FvBXwKP5119FxG9GxI/HH14FvA+WmdVE1fthddsL65cl/RnwHmAPcG1EfKuUyKrifbDMrCaq3g+rWw3kOrIpq++TdYxfNfaIquZ9sMysJqreD0udFjVJ+n5E/Fp++yDguxHxpnJCG8zMzEzMzs5WHYaZWa1Iui8iZvr5M91GIC80buSrpszMzIDuq7B+XVKjE13AK/L7AiIiXjXW6MzMLFkdE0hEHFhWIGZmVi/dRiDTbfduuPZa2LABFi2qOhozs33MzWVNg9u2ZUt3164t9/NBun4m+lTzdiZmlqiqtzEBJ5D2vJ2JmSWs6m1MwAmkvebtTNxIaGaJ2bYt27akWZnbmIATSDFvZ2Jmiat6GxNwAinm7UzMLHFVb2MCTiDFvJ2JmSWu6m1MoMtWJnXkrUzMzPo3jq1MzMzMCjmBmJnZQJxAzMxsIJUkEEmHSfqmpIfy74vbnPcNSc9IuqXsGF+2ezdceaWX8JpZUubm4NJL4dxzs+9ldqA3VDUCuRi4MyKWA3fm94t8AvjD0qIq4u1MplON3jikcCGxcqWwjQlUl0DWkH3aIfn3dxWdFBF3ArvLCmo/3s5ketXkjUMqFxIrVwrbmEB1CeSIiHgiv/1j4IhhnkzSekmzkmbn5+eHj67B25lMpxq9cUjlQmLlSmEbExhjApF0h6QHCr7WNJ8XWSPKUM0oEbExImYiYmbJkiVDxf0yb2cyvWr0xiGVC4mVK4VtTGCMCSQiTo+Ikwq+bgaelHQUQP79J+OKY2DezmQ61eyNQyoXEitXCtuYQHVTWJuAdfntdcDNFcXRnrczmU41e+OQyoXEypXCNiZQ0VYmkl4N3AAcCzwGvDcidkqaAS6IiPPz8+4G/g2wCHgKOC8ibu/03N7KxIZy+eXw8MP7Hz/hBLjkkvLj6UHVn0pnk2GQrUy8F5aZmXkvLDMzK89BVQdQC7t3w7XXwoYNsGhR1dHYqEzB79XTW5Mpld+rRyC9qElTmfVpwn+vbjKcTCn9Xp1AuqlRU5n1YQp+r24ynEwp/V6dQLqpUVOZ9WEKfq9uMpxMKf1enUA6qVlTmfVoSn6vbjKcTCn9Xp1AOqlZU5n1aEp+r24ynEwp/V6dQDpxN/pkmpLfayrdyjZaKf1e3UhoZmZuJDQzs/I4gZiZ2UDciW6TaQq6zEchlY5mqyePQPpRo8/JnnoT3mU+Cil1NFtv5uayz70/99zse9W/KyeQfviiVA9T0GU+Cil1NFt3KSZ8J5Be+aJUH1PQZT4KKXU0W3cpJnwnkF75olQPU9JlPgopdTRbdykmfCeQXviiVB9T0mU+Cil1NFt3KSb8ShKIpMMkfVPSQ/n3xQXnrJL0HUlbJM1Jel8VsQK+KNXJlHSZj0JKHc3WXYoJv6rPRL8S2BkRV0i6GFgcEX/Rcs6JQETEQ5JeA9wHvCEinun03GPpRK/h52Sb2eQZ57Lr2nwmuqQHgXdExBOSjgK+HRGv7/JnvgecHREPdTrPW5mYmfVvkARSVSPhERHxRH77x8ARnU6WdDKwACgYBthUcqNgadxsaO2MrQYi6Q5JDxR8rWk+L7IhUNthUD5C+RLwRxHxUptz1kualTQ7Pz8/0p+jkBsKq+eenFKk2HswjVJrIGwYWwKJiNMj4qSCr5uBJ/PE0EgQPyl6DkmvAm4FPhoRmzu81saImImImSVLlozjx9mXL17Vck9OaVLsPZg2KSfxqpbxbgLW5bfXATe3niBpAXAT8MWI+FqJsXXmi1f13JNTmhR7D6ZNykm8qgRyBfBOSQ8Bp+f3kTQj6XP5Oe8F3g58SNL9+deqasJt4otXtdyTU6oUew+mTcpJvJIEEhFPRcRpEbE8n+ramR+fjYjz89v/IyJ+KSJWNX1Vu5jfF6/quSenVCn2HkyblJO4O9H74YtX9dwoWCo3G1Yv5STuj7TthxsKzawCZSylrk0j4Ti5kdDMrH91aiQ068yNgrXhRsPp5RrIsNxUOB7utamFlHsU6i7V5sFmTiDD8oVu9NxrUxsp9yjUWV0SsxPIMHyhGw/32tRGyj0KdVaXxOwEMgxf6EbPvTa1knKPQp3VJTE7gQzKF7rxcK9NraTco1BndUnMTiCD8oVuPNwoWCtuNByPuiRm94EMyk2FZjZGZS+PdiMhbiSsBfd4TB33iqRvkATiKaxRc19Id176PFXqsiS1anXo+2jlBDJqvjh25qXPU6cuS1KrVNck6wQySr44duelz1OnLktSq1TXJOsEMkq+OHbmpc9TqS5LUqtU1yTrBDIqvjh256XPU6kuS1KrVNck6wQyKr44ducej6nkXpHu6ppkK1nGK+kw4CvAMuBR4L0R8XTLOccBN5EluV8C/ntEfKbbc1e2jLeoL+SFF+CnP4Wvf326lqt6ma71aVqX+Tb/3AsXZu+n9uyp5u+gNn0gkq4EdkbEFZIuBhZHxF+0nLMgj+95SYuAB4DfiogfdXrupPpANm2Cv/1bWL8ezjqr6mjKM60/tw2ksQJp8eJs3n/Xruzd96SPUlL7uevUB7IGuC6/fR3wrtYTImJPRDyf311I3abbpnVF1rT+3Dawuq5AGtYk/NxVXZSPiIgn8ts/Bo4oOknSUklzwOPAx7uNPpIyrSuypvXntoHVdQXSsCbh5x5bApF0h6QHCr7WNJ8X2Rxa4TxaRDweESuB1wHrJLVLNOslzUqanZ+fH/nP0rdpXZE1rT+3DaWuK5CGNQk/99gSSEScHhEnFXzdDDwp6SiA/PtPujzXj8hqIKe0eXxjRMxExMySJUtG/aP0bxpWZBVt2TINP7eNXLcVSHXc4qMXdV151ayqKaxNwLr89jrg5tYTJB0j6RX57cXAbwMPlhbhMDotV52UvbKKtmzxMl0bQKdlvnXd4qNIayKE+i9vrmoV1quBG4BjgcfIlvHulDQDXBAR50t6J3A12fSWgE9HxMZuz53UKqwik7BCaffu7F/6okVZIrz6ai/XtbG49NIsaSxevPdY437jIlwHqa24KlKbVVgR8VREnBYRy/Oprp358dmIOD+//c2IWBkRv55/75o8kjcpK5RcKLeSTEKhGSZjxVWRei2NrbtJuPC6UG4lmoRCM0xOImzlBFKWThfeFOsi7WJyodxK1EuhOcUie2tMCxdORiJs5QRSlk4X3hQ/Q6RdTC6UW4m67aOVYpG9KKbHH4dHHqn3iqsiB1UdwNRovvA2u+cemJ/fWxc59dTqC9KttZrmmPx571aylSvbF5qbawuw9/uNN1ZXnC6K6YQT4Pnns9uN/b7OOy+dAvqgnEDK0u7Cu2kT3HRTVhfZuTN7x1/W6qx2mx4212rKjsmsD9u2Ze/ymzXXFqrYpLFdTNu312vlWC88hVWlqusiRdNULpJbjXQqspc1vdVa71iwYDLrHUWcQKpURl2kXSJqt6TYRXKrkU5F9m5LZ0dRfC9KUjt2ZJ/sMGn1jiJOIFVqV5C+5572/SKdEkK7VVNFiajdkmIXya1GOhXZOy2d7TQ6aZdYio4XJanXvhaWLq13h3mvKulEH6fkO9F70aiLHHccPPZY9talUYNo18ledLxdx3jj+GGHZSOM55/Pah3uKLcJ0qmLvfV24/6ePfDzn+/fMb56dfZfrPX4s89mieGAprfiL72UJY7Pf76UH3NkatOJbh10q4sUjUy6TUe1jjI8TWVToNP0VrvRyebNxdNen/508fFnnpmeekcRJ5DUdKuLdEoIzcc7JSJPU9kU6DS91a74HlGcWHbsKD5+6KH131F3GF7Gm5pu/SKtCWFmpjhRPPdc+0TkXg6bEu16SNauzWoesO+U1Nvelt1untratQuOPrr4+KpVewv2k9Tf0SvXQOqiuS7S8Nhj2b/sHTv2P/7CC3BQwfuDE05wAjGjuEcEinfNbVcDmaTi+CA1EI9A6qLdyOQ738mWfLQef/3rnSjMOmg3OrnoouIRxYknTu9Iox2PQMzMzKuwzMysPE4gZmY2ECcQMzMbSCUJRNJhkr4p6aH8++IO575K0nZJny4zRjMz66yqEcjFwJ0RsRy4M7/fzuXA/y4lKjMz61lVCWQNcF1++zrgXUUnSXozcATwv0qKy8zMelRVAjkiIp7Ib/+YLEnsQ9IBwNXARd2eTNJ6SbOSZufn50cbqZmZFRpbI6GkO4AjCx76aPOdiAhJRc0oG4DbImK7pI6vFREbgY2Q9YEMFrGZmfVjbAkkIk5v95ikJyUdFRFPSDoK+EnBab8JnCJpA7AIWCDpZxHRqV5iZmYlqaQTXdIngKci4gpJFwOHRcR/7HD+h4CZiLiwh+eeBx5rOXw48NMhQi6DYxwNxzgajnE06hTjcRGxpJ8/WNVeWFcAN0g6j+xi/14ASTPABRFx/qBPXPQXIGm23xb9sjnG0XCMo+EYR2PSY6wkgUTEU8BpBcdngf2SR0R8AfjC2AMzM7OeuRPdzMwGMi0JZGPVAfTAMY6GYxwNxzgaEx3jxG3nbmZm5ZiWEYiZmY3YRCaQOmzW2EuMko6T9I+S7pe0RdIFCca4StJ38vjmJL0vtRjz874h6RlJt5QY2xmSHpS0NV+u3vr4QklfyR+/R9KysmLrI8a35/8GX5R0dtnx9RjjRyT9IP/3d6ek44qep+IYL5D0/fz/8v+RtCK1GJvOe4+kyFfFdhYRE/cFXAlcnN++GPh4h3M/BXwZ+HRqMQILgIX57UXAo8BrEovxRGB5fvs1wBPAoSnFmD92GnAWcEtJcR0IPAy8Nv89fg9Y0XLOBuAz+e1zgK+U/G+wlxiXASuBLwJnlxlfHzGeCvxKfvuPE/17fFXT7dXAN1KLMT/vYLLNazeT9d51fN6JHIFQj80au8YYEXsi4vn87kLKHzH2EuM/RcRD+e0fke0q0Fcz0pB6+l1HxJ3A7rKCAk4GtkbEIxGxB7ieLNZmzbF/DThN3fbtKTnGiHg0IuaAl0qMq1kvMd4VEc/ldzcDxyQY47NNd18JlF187uXfI2S7n38c+JdennRSE8hIN2sck64xAkhaKmkOeJzs3fWPygqQHmNskHQy2bubh8cdWJO+YizR0WS/s4bt+bHCcyLiRWAX8OpSomt5/VxRjFXrN8bzgH8Ya0T76ylGSR+W9DDZqPlPSoqtoWuMkt4ELI2IW3t90qo60YdW5maNgxpBjETE48BKSa8B/qekr0XEkynFmD/PUcCXgHURMdJ3q6OK0SabpA8CM8DvVB1LkYi4BrhG0geAvwTWVRzSy/I31J8EPtTPn6ttAokabNY4ghibn+tHkh4ATiGb7kgmRkmvAm4FPhoRm0cV2yhjrMAOYGnT/WPyY0XnbJd0EHAI8FQ54e3z+g1FMVatpxglnU72huJ3mqZ9y9Lv3+P1wLVjjWh/3WI8GDgJ+Hb+hvpIYJOk1ZHtEFJoUqewNrE3u68Dbm49ISL+ICKOjYhlZNNYXxxl8uhB1xglHSPpFfntxcBvAw+WFmFvMS4AbiL7+xtZYutD1xgrci+wXNLx+d/ROWSxNmuO/WzgW5FXMhOKsWpdY5T0G8BngdURUcUbiF5iXN509/eAh0qMD7rEGBG7IuLwiFiWXxM3k/19tk0ejT8wgD+cAAAB0klEQVQ4cV9k88h3kv2S7iDb7Rey4e3nCs7/EOWvwuoaI/BOYI5sxcQcsD7BGD8IvADc3/S1KqUY8/t3A/PAL8jmf3+3hNj+LfBPZDWhj+bHLsv/YwL8MvBVYCvwXeC1Zf5+e4zxLfnf18/JRkdbEozxDuDJpn9/mxKM8VPAljy+u4A3phZjy7nfpodVWO5ENzOzgUzqFJaZmY2ZE4iZmQ3ECcTMzAbiBGJmZgNxAjEzs4E4gZiNiKR/zXdbfUDSVyX9Sn78SEnXS3pY0n2SbpN0Yv5Y6bsEm42KE4jZ6PwiIlZFxEnAHuCCfHPEm4BvR8QJEfFm4D+xd8+uTwB/WE24ZsNxAjEbj7uB15FtNf5CRHym8UBEfC8i7s5vl71LsNnIOIGYjVi+r9WZwPfJ9he6r9qIzMbDCcRsdF4h6X5gFtgG/F3F8ZiNVW134zVL0C8iYlXzAUlbyDZKNJs4HoGYjde3gIWS1jcOSFop6ZQKYzIbCScQszGKbLfSdwOn58t4twD/lezTE5F0N9mOvKdJ2i7pd6uL1qw/3o3XzMwG4hGImZkNxAnEzMwG4gRiZmYDcQIxM7OBOIGYmdlAnEDMzGwgTiBmZjYQJxAzMxvI/wcC0+McjJxxzgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff3a50afe90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(X_skernpca[y==0, 0], X_skernpca[y==0, 1],\n",
    "           color='red', marker='^', alpha=0.5)\n",
    "plt.scatter(X_skernpca[y==1, 0], X_skernpca[y==1, 1],\n",
    "           color='blue', marker='o', alpha=0.5)\n",
    "plt.xlabel('PC1')\n",
    "plt.ylabel('PC2')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 总结\n",
    "\n",
    "本章，你学习了三种基本的用于特征抽取的降维方法：标准PCA、LDA和核PCA。\n",
    "\n",
    "使用PCA，我们将数据映射到一个低维度的子空间并且最大化正交特征轴的方差，PCA不考虑类别信息。LDA是一种监督降维方法，意味着他要考虑训练集的类别信息，目标是将类别最大化地可分。最后，你学习了核PCA，它能够将非线性数据集映射到低维特征空间，然后数据变成线性可分了。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
